Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Package: link
Title: Stream Network Habitat Interpretation (Experimental)
Version: 0.40.3
Version: 0.40.4
Date: 2026-05-23
Authors@R: c(
person("Allan", "Irvine", , "airvine@newgraphenvironment.com",
Expand Down
6 changes: 6 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
# link 0.40.4

Reproduce bcfp's per-species accessibility so dam-downstream segments emit the dam descriptor ([#200](https://github.com/NewGraphEnvironment/link/issues/200)). The mapping_code phase previously drove `accessible` from `barriers_<sp>_unified` (all barriers, including dams), so every segment below a dam read inaccessible and lost its `;DAM`/`;MODELLED`/`;ASSESSED` second token — emitting a bare `SPAWN`/`REAR` where bcfp emits `SPAWN;DAM`. It now uses a new per-species `barriers_<sp>_access` view that reproduces bcfp's `barriers_<sp>`: natural barriers only (gradient at the species threshold ∪ falls ∪ subsurface), minus the observation/habitat override, plus all user-definite barriers (override-exempt). Dams stay in `barrier_sources` and annotate token2 only.

All three access inputs are now persisted province-wide so the cross-WSG downstream walk is correct in every watershed group, not just the run's own: natural barriers (already), `user_barriers_definite` (new `USER_DEFINITE` family in `lnk_barriers_unify`, ltree-resolved via the FWA join like falls), and the observation/habitat override (new `<persist_schema>.barrier_overrides` table). Validated against `bcfishpass@v0.7.15`: PARS BT 98.95%, LFRA BT 97.77% / CO 97.90% per-segment mapping_code match. See `RUNBOOK.md` §5.

# link 0.40.3

Persist the per-source downstream-barrier flag columns in `streams_access` so `lnk_pipeline_mapping_code`'s second token (`DAM`/`MODELLED`/`ASSESSED`/`REMEDIATED`/`NONE`) populates from persisted state instead of defaulting to `NONE`. Three coupled fixes ([#196](https://github.com/NewGraphEnvironment/link/issues/196)): `lnk_persist_init` adds the six flag columns to the `streams_access` DDL; `lnk_pipeline_run` pre-persists barriers before the mapping_code phase for cross-WSG dam visibility (link#152); `lnk_pipeline_persist` projects the flag columns in the INSERT (the DDL/INSERT pair must match — the missing projection was the actual `NONE`-token bug).
Expand Down
45 changes: 44 additions & 1 deletion R/lnk_barriers_unify.R
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,13 @@
#' - **Subsurface_flow** (`barrier_source = 'SUBSURFACE_FLOW'`, from
#' `<schema>.barriers_subsurfaceflow`): blocks all 8 species. Opt-in
#' (only built when `cfg$pipeline$break_order` includes `"subsurfaceflow"`).
#' - **User_definite** (`barrier_source = 'USER_DEFINITE'`, from
#' `<schema>.barriers_definite`): blocks all 8 species. ltrees +
#' `linear_feature_id` resolved by JOIN to
#' `whse_basemapping.fwa_stream_networks_sp` (mirrors the FALLS branch
#' and bcfp's `barriers_user_definite.sql`). Persisted province-wide so
#' the per-species access view (`barriers_<sp>_access`,
#' [lnk_barriers_views()]) can include them override-exempt — link#200.
#'
#' Remediations (PASSABLE remediation crossings) are intentionally NOT
#' in this table. They're consumed via `<schema>.barriers_remediations`
Expand Down Expand Up @@ -67,6 +74,8 @@
#' - `<schema>.crossings` (from [lnk_pipeline_crossings()]).
#' - `<schema>.gradient_barriers_raw` (from [lnk_pipeline_prepare()]).
#' - `<schema>.falls` (from [lnk_pipeline_prepare()]).
#' - `<schema>.barriers_definite` (from [lnk_pipeline_prepare()]; always
#' created, may be empty).
#'
#' Optional:
#' - `<schema>.barriers_subsurfaceflow` (only when the config opts in).
Expand Down Expand Up @@ -241,7 +250,41 @@ lnk_barriers_unify <- function(conn, aoi, cfg, loaded,
WHERE f.watershed_group_code = %3$s",
all_species_arr, schema, aoi_lit) # nolint: indentation_linter

union_parts <- c(sql_anth, sql_gradient, sql_falls)
# Source 4: user-definite barriers. id_barrier = 'USER_DEFINITE-' ||
# row_number(). <schema>.barriers_definite (built in lnk_pipeline_prepare
# from user_barriers_definite.csv) carries blue_line_key +
# downstream_route_measure only — the empty-fallback shape has just those
# two columns — so reference ONLY those from `d` and JOIN to FWA for
# ltrees + linear_feature_id + watershed_key, exactly as the FALLS branch
# does. Mirrors bcfp model/01_access/sql/barriers_user_definite.sql.
# blocks_species = all (a user-definite barrier blocks every species).
# These are override-EXEMPT: the per-species access view
# (barriers_<sp>_access) never lifts them via barrier_overrides — bcfp
# unions all user_definite AFTER the observation/habitat filter ("include
# *all* user added features, even those below observations"). link#200.
sql_user_definite <- sprintf("
SELECT
('USER_DEFINITE-' || row_number() OVER ()::text)::text AS id_barrier,
%1$s::varchar(4) AS watershed_group_code,
'USER_DEFINITE'::text AS barrier_source,
'user_definite'::text AS barrier_subtype,
'BARRIER'::text AS passability,
%2$s AS blocks_species,
s.linear_feature_id,
d.blue_line_key,
s.watershed_key,
d.downstream_route_measure::double precision,
s.wscode_ltree,
s.localcode_ltree,
ST_Force2D(FWA_LocateAlong(d.blue_line_key, d.downstream_route_measure))::geometry(Point, 3005) AS geom
FROM %3$s.barriers_definite d
JOIN whse_basemapping.fwa_stream_networks_sp s
ON d.blue_line_key = s.blue_line_key
AND d.downstream_route_measure >= s.downstream_route_measure
AND d.downstream_route_measure < s.upstream_route_measure",
aoi_lit, all_species_arr, schema) # nolint: indentation_linter

union_parts <- c(sql_anth, sql_gradient, sql_falls, sql_user_definite)

if (has_subsurface) {
sql_subsurface <- sprintf("
Expand Down
73 changes: 65 additions & 8 deletions R/lnk_barriers_views.R
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,24 @@
#' Fixes the PARS BT 60% defect (PARS drains through dams in PCEA /
#' UPCE WSGs) and unblocks any regional run.
#'
#' Per-species views:
#' - `<schema>.barriers_<sp>_unified` for each species (`bt`, `ch`, `cm`,
#' `co`, `pk`, `sk`, `st`, `wct`) — filtered by
#' `'<SP>' = ANY(blocks_species)`. `_unified` suffix avoids name
#' collision with the per-WSG `<schema>.barriers_<sp>` tables that
#' `.lnk_pipeline_prep_minimal` builds for the break-time path.
#' Per-species views (two per species `bt`, `ch`, `cm`, `co`, `pk`, `sk`,
#' `st`, `wct`):
#' - `<schema>.barriers_<sp>_unified` — filtered by
#' `'<SP>' = ANY(blocks_species)` (ALL barrier families incl dams).
#' `_unified` suffix avoids name collision with the per-WSG
#' `<schema>.barriers_<sp>` tables that `.lnk_pipeline_prep_minimal`
#' builds for the break-time path.
#' - `<schema>.barriers_<sp>_access` — the per-species **accessibility**
#' set that drives `accessible` in mapping_code (link#200). Reproduces
#' bcfp `barriers_<sp>`: NATURAL barriers only
#' (`barrier_source IN ('GRADIENT','FALLS','SUBSURFACE_FLOW')` — the
#' gradient-at-species-threshold is already encoded in `blocks_species`)
#' MINUS the observation/habitat override (anti-join `barrier_overrides`),
#' ∪ all `USER_DEFINITE` (override-exempt). Dams/PSCIS/modelled are
#' excluded — they annotate token2 via the per-source views, never block
#' access. Same feature shape (`barriers_<sp>_access_id` + ltrees + geom)
#' so [lnk_pipeline_access()] / [fresh::frs_network_features()] consume it
#' unchanged. See `RUNBOOK.md` §5.
#'
#' Per-source views (matching the bcfp source-typed tables consumed by
#' the `barrier_sources` arg of `lnk_pipeline_access`):
Expand Down Expand Up @@ -47,8 +59,9 @@
#' either way (the underlying table has `blocks_species` from
#' [lnk_barriers_unify()]).
#'
#' @return `invisible(conn)`. Side effect: drops + recreates one view
#' per species + three source-typed views in `schema`.
#' @return `invisible(conn)`. Side effect: drops + recreates two views
#' per species (`_unified` + `_access`) + three source-typed views in
#' `schema`.
#'
#' @details
#' Views are dropped + recreated on each call (`CREATE OR REPLACE VIEW`)
Expand Down Expand Up @@ -112,6 +125,13 @@ lnk_barriers_views <- function(conn, schema, cfg,
paste0(tn$schema, ".barriers")
}

# The per-species access view (barriers_<sp>_access) anti-joins the
# barrier-overrides skip list. Read it from the SAME schema as the
# barriers source so persist-barriers pairs with the province-wide
# overrides (link#200) and a working-schema barriers_table (if ever
# passed) pairs with that schema's per-WSG overrides. link#200.
overrides_table <- sub("\\.barriers$", ".barrier_overrides", persist_barriers)

# Per-species views. Each view re-exposes id_barrier as
# `barriers_<sp>_unified_id` so fresh::frs_network_features's
# `feature_id_col = "<table>_id"` convention works unchanged.
Expand All @@ -137,6 +157,43 @@ lnk_barriers_views <- function(conn, schema, cfg,
)
.lnk_db_execute(conn, sprintf("DROP VIEW IF EXISTS %s", view_name))
.lnk_db_execute(conn, sql_view)

# Per-species ACCESS view (link#200). The set that drives `accessible`
# in mapping_code — reproduces bcfp `barriers_<sp>`: NATURAL barriers
# only (gradient at the species threshold — already encoded in
# blocks_species — ∪ falls ∪ subsurface) MINUS the observation/habitat
# override, ∪ all user_definite (override-EXEMPT). Dams/PSCIS/modelled
# are NOT here (they annotate token2 via barrier_sources, never block
# access). Same feature shape as `_unified` (id + ltrees + geom) so
# fresh::frs_network_features walks it unchanged. The override anti-join
# reads province-wide `overrides_table` so a natural barrier in ANY WSG
# a downstream walk crosses is lifted correctly. See RUNBOOK.md §5.
access_view <- paste0(schema, ".barriers_", sp_lower, "_access")
access_id <- paste0("barriers_", sp_lower, "_access_id")
sql_access <- sprintf(
"CREATE OR REPLACE VIEW %s AS
SELECT id_barrier AS %s,
barrier_source, barrier_subtype, passability,
blocks_species,
linear_feature_id, blue_line_key, watershed_key,
downstream_route_measure, wscode_ltree, localcode_ltree,
watershed_group_code, geom
FROM %s b
WHERE %s = ANY(blocks_species)
AND barrier_source IN ('GRADIENT', 'FALLS', 'SUBSURFACE_FLOW', 'USER_DEFINITE')
AND (
barrier_source = 'USER_DEFINITE'
OR NOT EXISTS (
SELECT 1 FROM %s o
WHERE o.species_code = %s
AND o.blue_line_key = b.blue_line_key
AND abs(o.downstream_route_measure - b.downstream_route_measure) < 1
)
)",
access_view, access_id, persist_barriers, sp_lit, overrides_table, sp_lit
)
.lnk_db_execute(conn, sprintf("DROP VIEW IF EXISTS %s", access_view))
.lnk_db_execute(conn, sql_access)
}

# Per-source views — unified (cross-WSG) shape exposed under a
Expand Down
41 changes: 41 additions & 0 deletions R/lnk_persist_init.R
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,24 @@ cols_barriers <- c(
geom = "geometry(Point, 3005)"
)

#' Column shape for the province-wide barrier-overrides table.
#'
#' `<persist_schema>.barrier_overrides` holds the per-(segment x species)
#' observation/habitat barrier-skip list (`lnk_barrier_overrides` output)
#' accumulated across all WSGs. Persisted province-wide so the per-species
#' access view (`barriers_<sp>_access`, [lnk_barriers_views()]) can
#' anti-join it for natural barriers in ANY WSG a downstream walk crosses —
#' the cross-WSG twin of the link#152 barriers fix. One named vector drives
#' both the DDL ([lnk_persist_init()]) and the INSERT projection
#' ([lnk_pipeline_persist()]). link#200.
#' @noRd
cols_barrier_overrides <- c(
blue_line_key = "integer NOT NULL",
downstream_route_measure = "double precision NOT NULL",
species_code = "text NOT NULL",
watershed_group_code = "varchar(4) NOT NULL"
)

#' Validate that an existing target table doesn't carry stale DDL
#' (unexpected `GENERATED ALWAYS` columns).
#'
Expand Down Expand Up @@ -309,6 +327,8 @@ lnk_persist_init <- function(conn, cfg, species, force_recreate = FALSE) {
}
.lnk_validate_persist_table(conn, schema = schema, table = "barriers",
force_recreate = force_recreate)
.lnk_validate_persist_table(conn, schema = schema, table = "barrier_overrides",
force_recreate = force_recreate)
.lnk_validate_persist_table(conn, schema = schema, table = "streams_access",
force_recreate = force_recreate)
.lnk_validate_persist_table(conn, schema = schema,
Expand Down Expand Up @@ -375,6 +395,27 @@ lnk_persist_init <- function(conn, cfg, species, force_recreate = FALSE) {
idx_name, schema, barriers_idx_specs[[idx_name]]))
}

# Province-wide barrier-overrides table (link#200). The per-(segment x
# species) observation/habitat barrier-skip list, accumulated per-WSG so
# the per-species access view can anti-join it cross-WSG. PK includes
# watershed_group_code (mirrors cols_barriers' (id_barrier, wsg) PK):
# the SAME override position can be computed by two adjacent WSG runs
# (boundary streams whose blue_line_key spans WSGs), so (blk, drm,
# species) is NOT unique across WSGs. Per-WSG DELETE-WHERE-WSG + INSERT
# stays clean; the access anti-join is WSG-agnostic so duplicates are
# harmless.
barrier_overrides_pk <- c("blue_line_key", "downstream_route_measure",
"species_code", "watershed_group_code")
.lnk_db_execute(conn, sprintf(
"CREATE TABLE IF NOT EXISTS %s.barrier_overrides (\n %s\n)",
schema, .lnk_cols_clause(cols_barrier_overrides, barrier_overrides_pk)))
.lnk_db_execute(conn, sprintf(
"CREATE INDEX IF NOT EXISTS barrier_overrides_wsg_idx ON %s.barrier_overrides (watershed_group_code)",
schema))
.lnk_db_execute(conn, sprintf(
"CREATE INDEX IF NOT EXISTS barrier_overrides_blk_drm_idx ON %s.barrier_overrides (blue_line_key, downstream_route_measure)",
schema))

# Per-segment per-species access table (link#187). Wide shape (one
# row per segment, all species columns inline) — matches the QGIS-
# consumer expectation + bcfp's `bcfishpass.streams_access` shape.
Expand Down
32 changes: 32 additions & 0 deletions R/lnk_pipeline_persist.R
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,38 @@ lnk_pipeline_persist <- function(conn, aoi, cfg, species,
barriers_table, barriers_cols, barriers_cols, schema))
}

# ----- barrier_overrides (link#200) -----
# Per-(segment x species) observation/habitat skip list. Working
# <schema>.barrier_overrides (built by lnk_barrier_overrides in
# lnk_pipeline_prepare) carries (blue_line_key, downstream_route_measure,
# species_code) only — add the AOI as watershed_group_code in the SELECT.
# Persisted province-wide so barriers_<sp>_access can anti-join it for
# natural barriers in any WSG a downstream walk crosses. Probe-gated like
# barriers above.
overrides_present <- nrow(DBI::dbGetQuery(conn, sprintf(
"SELECT 1 FROM information_schema.tables
WHERE table_schema = %s AND table_name = 'barrier_overrides'
LIMIT 1;",
DBI::dbQuoteString(conn, schema)
))) > 0L
if (overrides_present) {
overrides_table <- paste0(tn$schema, ".barrier_overrides")
overrides_cols <- paste(names(cols_barrier_overrides), collapse = ", ")
# SELECT projection: WSG is the literal AOI (working table lacks it);
# the other three come straight from the working override table.
overrides_select <- vapply(names(cols_barrier_overrides), function(col) {
if (col == "watershed_group_code") paste0(aoi_lit, "::varchar(4)")
else col
}, character(1))
.lnk_db_execute(conn, sprintf(
"DELETE FROM %s WHERE watershed_group_code = %s",
overrides_table, aoi_lit))
.lnk_db_execute(conn, sprintf(
"INSERT INTO %s (%s) SELECT %s FROM %s.barrier_overrides",
overrides_table, overrides_cols,
paste(overrides_select, collapse = ", "), schema))
}

# ----- streams_access (link#187) -----
# Per-segment per-species access. Working `<schema>.streams_access` is
# built by lnk_pipeline_access; persist phase copies the per-WSG slice
Expand Down
27 changes: 13 additions & 14 deletions R/lnk_pipeline_run.R
Original file line number Diff line number Diff line change
Expand Up @@ -198,22 +198,21 @@ lnk_pipeline_run <- function(conn, aoi, cfg, loaded,
species = active_species)

# 2. Per-segment access. barriers_per_sp drives `accessible` (token1
# ACCESS + token2 gate). KNOWN-DIVERGENT from bcfp — see RUNBOOK.md §5.
# bcfp's per-species access set (`barriers_<sp>`, built in
# smnorris/bcfishpass model/01_access/sql/model_access_bt.sql) is
# NATURAL barriers only (gradient at the species threshold + falls +
# subsurface), MINUS barriers with upstream observations / confirmed
# habitat, plus user_definite — dams are NEVER in it (they annotate
# access via token2, not block it). link's `_unified` views instead
# carry ALL barriers incl dams (blocks_species universal, §2a), so
# dam-downstream segments read inaccessible and lose their `;DAM`
# token. The correct fix is a natural-only per-species *feature* view
# with the override applied (reproducing bcfp barriers_<sp>) — real
# work, scoped separately, not a table swap. `_min` cannot be used
# here: it is a break-spec with no id column (RUNBOOK §2b).
# ACCESS + token2 gate). Uses the per-species `barriers_<sp>_access`
# views (link#200) — these reproduce bcfp's per-species access set
# (`barriers_<sp>`, smnorris/bcfishpass model/01_access/sql/
# model_access_bt.sql): NATURAL barriers only (gradient at the species
# threshold + falls + subsurface), MINUS the observation/habitat
# override, ∪ user_definite (override-exempt). Dams/PSCIS/modelled are
# NOT in the access set — they annotate access via token2
# (barrier_sources below), never block it. So a dam-downstream segment
# that is otherwise accessible stays accessible and emits `;DAM`. All
# inputs are province-wide-persisted (barriers + barrier_overrides via
# the pre-persist above), so the cross-WSG downstream walk is correct in
# every WSG, not just the run's own. See RUNBOOK.md §5.
sp_set <- tolower(active_species)
barriers_per_sp <- setNames(
lapply(sp_set, function(sp) paste0(schema, ".barriers_", sp, "_unified")),
lapply(sp_set, function(sp) paste0(schema, ".barriers_", sp, "_access")),
sp_set)

lnk_pipeline_access(conn, # nolint: object_usage_linter
Expand Down
Loading