Skip to content

feat(dag-viewer): show planner cost and benefit annotations - #296

Merged
zzylol merged 15 commits into
feat/cost-streaming-linearfrom
feat/dag-viewer-cost-annotations-286
Sep 3, 2026
Merged

zzylol merged 15 commits into
feat/cost-streaming-linearfrom
feat/dag-viewer-cost-annotations-286

Conversation

@zzylol

@zzylol zzylol commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Closes #286.

Why

The planner now produces evidence-backed CPU, peak-memory, and source-I/O comparisons, but those decisions are difficult to audit from a raw plan alone. The export and viewer need to show the selected physical alternative, its equally scoped raw baseline, the resulting benefit, and the evidence provenance without recomputing costs in JavaScript.

This PR is intentionally the top of the analytical-cost stack: it consumes the completed planner/streaming cost contracts and adds presentation. It does not define planner formulas or structural ranking.

What

  • Adds structured cost annotations with units, source, baseline, delta, benefit ratio, model version, benchmark identity, and inputs.
  • Exports baseline, selected, and benefit annotations for the exact candidate chosen by whole-plan global_selection.
  • Adds per-query and workload-wide summaries with explicit deduplication identity.
  • Annotates genuine shared-DAG edges without inventing multi-hop path costs.
  • Renders graph badges plus node, edge, decision, lifecycle, and workload cost details.
  • Adds a complete physical-evidence input contract through --planner-cost-json.
  • Keeps the output schema backward compatible by making annotation fields optional.

How

The planner evidence document contains calibration, an exact target QueryExpr, comparison scope, exact query-node selectors with PhysicalNodeEvidence, and candidate-specific evidence. Logical rewrites are recursively lowered from their query-node evidence; summary replacements carry a complete bound PhysicalDag. Target, query-node, and candidate matching uses full structural equality, not a hash or strategy name.

Candidate generation is bound to the supplied analytical model, and final selection uses PlanSpace::global_selection. Viewer annotations are produced from the selected candidate's same complete raw-versus-candidate comparison. The exporter does not use DefaultCostModel, structural node counts, or a Top-K query shortcut in production.

Unknown fields, duplicate selectors, missing evidence, unused target/candidate/query evidence, invalid physical DAGs, conflicting identities, unsupported algorithms, or incomplete source coverage fail closed. Without a complete evidence document, --post-asap exports the raw graph only. The old --analytical-cost-json spelling accepts the new document as an alias; its former compact aggregation payload produces an explicit migration error.

group_count belongs only to aggregate operator evidence, while k belongs only to a Top-K operator. Neither is promoted into a global viewer/export input.

Before this PR

Cost explanations were not attached to the exported DAG. Earlier versions of this branch used relative structural units, but that production path and its generated fixture have been removed.

After this PR

The viewer displays only explicit modeled, measured, or unavailable annotations. It never estimates costs client-side. A complete physical evidence document can select and explain generic non-Top-K, relational, PromQL, shared-DAG, and streaming-summary alternatives; incomplete evidence preserves the raw plan.

Verification

  • cargo test -p asap-aware-mapping --all-targets — 313 passed.
  • cargo test -p asap-devtools --bin dag_export — 17 passed.
  • python3 -m unittest discover -s tools/dag-viewer -p "test_*.py" — 21 passed.
  • cargo clippy -p asap-aware-mapping -p asap-devtools --all-targets -- -D warnings — passed.
  • cargo fmt --all -- --check — passed.
  • git diff --check — passed.
  • Independent final correctness review — no remaining blocker.

The exporter tests include JSON-to-provider-to-planner-to-annotation coverage for a generic non-Top-K DAG, selection of the numerically cheapest of two complete candidates, and fail-closed duplicate, missing, unused, unknown-field, and invalid-DAG cases.

Viewer visualization

These captures demonstrate the pre/post lanes, decision badges, graph-element linkage, edge details, and workload comparison UI. They are presentation references only; the previous structural fixture shown in the captures is no longer generated or accepted as planner evidence. Current cost values must come from --planner-cost-json and are displayed with their explicit modeled units and provenance.

DAG viewer showing pre/post lanes, replacement regions, and edge cost details

DAG viewer showing the positive-benefit comparison layout

@zzylol zzylol changed the title feat(dag-viewer): export and visualize planner cost/benefit annotations (#286) feat(dag-viewer): show planner cost and benefit annotations Aug 29, 2026
@zzylol
zzylol force-pushed the feat/dag-viewer-cost-annotations-286 branch from 90f4139 to 369e6a0 Compare September 2, 2026 19:28
@zzylol
zzylol changed the base branch from main to feat/cost-streaming-linear September 2, 2026 19:29
@zzylol
zzylol force-pushed the feat/dag-viewer-cost-annotations-286 branch from 1b34eb3 to 7f95330 Compare September 3, 2026 02:54
@zzylol
zzylol force-pushed the feat/cost-streaming-linear branch from e16c007 to a6228d1 Compare September 3, 2026 02:54
@zzylol
zzylol force-pushed the feat/dag-viewer-cost-annotations-286 branch 3 times, most recently from 16b5e9a to 40aab73 Compare September 3, 2026 13:17
zzylol and others added 15 commits September 3, 2026 07:26
Adds a structured, optional CostAnnotation schema (crates/types/src/cost.rs)
and wires it through dag_export's JSON output and the tools/dag-viewer
sidebar/on-graph UI, per issue #286.

Rust:
- `asap_types::cost`: `CostAnnotation` (value/unit/source/baseline/delta/
  benefit_ratio/model_version/benchmark_id/inputs), `CostUnit`
  (CostUnitsPerSecond / CostUnits / RelativeStructuralUnits), `CostSource`
  (Modeled/Measured/Unavailable), `BaselineRef`, `CostInput`,
  `total_cost(rate, horizon, one_shot)`, `sum_workload_costs` (dedups by an
  explicit key, rejects unit-mismatched aggregation), and
  `WorkloadCostSummary`/`workload_cost_summary`.
- `DagGraph` gains `edge_annotations: Vec<EdgeCostAnnotation>`, populated by
  `deduplicate_pointer_shared_nodes` for every edge running into a genuine
  DAG merge point (never a guessed multi-hop path cost).
- `DagDecision` and `TargetReplacement` gain `baseline_cost`/`selected_cost`/
  `benefit` alongside their existing bare `cost: f64` (unchanged, for
  backward compat). `NamedGraph`/`WorkloadGraph` gain `workload_cost:
  Option<WorkloadCostSummary>`.
- crates/devtools/src/bin/dag_export.rs populates all of the above from
  today's `asap_aware_mapping::cost_model` output (`estimate_cost`,
  `default_cse_recompute_cost`), deduplicating workload totals by
  `decision.id`.

Every value dag_export produces today is honestly unit-tagged
`RelativeStructuralUnits` (the same structural-size proxy the cost model
already uses for ranking), not `CostUnitsPerSecond`: the cost model has no
update_rate/evaluation_rate/query_interval recurrence inputs yet (#287's
job). The annotation plumbing accepts a real rate unchanged once #287 lands
those inputs. Nothing is ever fabricated: a value the cost model can't
estimate is `CostSource::Unavailable` (`value: None`), never `0`.

JS/viewer:
- viewer.js renders concise on-graph `▼NN%`/`▲NN%` badges on a costed
  post-ASAP node's label, full baseline/selected/benefit/provenance blocks
  in the sidebar (node click, edge click, and a workload-scope cost summary
  for the current single- or multi-query selection), all sourced only from
  explicit JSON fields (decision.id dedup, `EdgeCostAnnotation`,
  `workload_cost`) — no client-side cost estimation.
- index.html: cost UI CSS (light/dark aware, via existing --var tokens).
- tools/dag-viewer/dag.example.json regenerated via generate-sample.sh
  (real lowering -> ASAP-aware mapping -> post-ASAP -> dag_export
  pipeline), not hand-patched.
- README.md documents the new JSON contract fields.

Tests: `cargo test --workspace` (all green) and
`python3 -m unittest discover -s tools/dag-viewer -p test_render.py`
(18/18) both pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Fixes 5 confirmed bugs from PR review plus 3 lower-priority follow-ups.

Confirmed bugs:

1. viewer.js: `loadFiles()`/`loadWorkload()` dropped `workload_cost` from
   the object pushed onto `queries` — the single-query "Workload cost"
   panel (renderScopeSummary's `selected[0].workload_cost` read) was
   silently `undefined` on every interactive load path (drag-and-drop,
   file picker, and the planner/embedded path), even though the exported
   JSON carried the data. Both loaders now forward `workload_cost`.

2. viewer.js: `computeSelectionWorkloadCost` deduped decisions across a
   multi-query selection by bare `decision.id`, which is only unique
   within one `dag_export` process invocation, not across independently
   loaded files — a real collision (two files reusing the same small
   integer id) would silently drop one file's cost from the aggregate.
   Added a `sourceBatch` id assigned once per loaded document/file and
   changed the dedup key to `${sourceBatch}:${decision.id}`.

3. dag_export.rs: `shared_node_edge_annotations` counted edge occurrences
   (`Vec`) rather than distinct consuming nodes as `consumer_count` — a
   `Join` whose left and right operands are the same `Rc` (post
   pointer-dedup) inflated `consumer_count` to 2 for one real downstream
   consumer, halving the reported per-edge cost and producing two
   colliding `(from, to)` `EdgeCostAnnotation` entries (which
   `edgeCostByPair` in viewer.js then silently collapsed via Map
   overwrite). Switched to a `HashSet` per child so a single parent
   referencing the same shared child twice counts as one consumer. Added
   a regression test plus updated the existing edge-annotation test to use
   two genuinely distinct parents.

4. cost.rs: `total_cost()` validated `horizon` but not
   `recurring_cost_rate`/`one_shot_cost` themselves — a non-finite rate
   (e.g. a stray NaN from a future #287 caller) silently produced
   `Some(NaN)` instead of `None`, violating the module's own "never
   fabricate, never a poisoned total" rule. Both inputs are now validated
   finite before use; added regression tests.

5. dag_export.rs: `NamedGraph.workload_cost`'s doc claimed cross-query
   dedup via `workload_node_id`, but the actual producer
   (`decision_cost_entries`) only dedups within one query by
   `decision.id` — a reader trusting the doc and summing
   `NamedGraph.workload_cost` across queries would double-count a target
   shared between them. Corrected the doc to state the per-query-only
   scope and point cross-query readers at `WorkloadGraph.workload_cost`
   instead (the implementation was already correct; only the doc was
   wrong).

Lower-priority follow-ups also addressed:

- dag_export.rs: the legacy scalar `cost: f64` on `DagDecision`/
  `TargetReplacement` is now derived from `selected_cost.value` at both
  call sites instead of being set independently from `winner.cost` a
  second time, closing the "kept in sync by convention only" gap the
  review flagged.
- dag_export.rs: `default_cse_recompute_cost` is now memoized once per
  winner (`per_consumer_recompute_costs`, built right after `winners`)
  instead of being recomputed on every `winner_cost_annotations` call —
  a winner's target can be reached from more than one node position
  (internal sharing within a query, or the same CSE-shared target across
  several queries), so this avoided redundant subtree walks.
- dag_export.rs (types crate): `shared_node_edge_annotations`'s
  `parents_of` map is now built inline inside
  `deduplicate_pointer_shared_nodes`'s existing per-node loop (which
  already visits every remapped child edge once while assigning final
  ids) instead of a second full pass over the deduplicated node list.

Not fixed (noted only): `computeSelectionWorkloadCost` in viewer.js still
hand-reimplements `cost.rs`'s `sum_workload_costs`/`workload_cost_summary`
dedup-and-sum algorithm in JS, with no shared source of truth — there's no
JS/Rust code-sharing mechanism in this tool today, so keeping the two
algorithms in sync remains a manual/review responsibility. Flagged as a
follow-up in the PR description.

Testing: `cargo build --workspace`, `cargo test --workspace` (all green,
no regressions), `cargo clippy --workspace --all-targets -- -D warnings`
(clean), `cargo fmt --all -- --check` (clean, after running `cargo fmt
--all` once for pre-existing drift), and
`python3 -m unittest discover -s tools/dag-viewer` (18/18). Regenerated
`dag.example.json` via generate-sample.sh — byte-identical, since the
sample workload doesn't happen to exercise the same-parent-twice edge
case fixed in item 3. `node --check` remains unavailable in this sandbox
(no Node.js installed); verified the viewer.js changes by careful manual
review plus the Python test suite, which inlines and structurally checks
viewer.js.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@zzylol
zzylol force-pushed the feat/dag-viewer-cost-annotations-286 branch from 40aab73 to b18f708 Compare September 3, 2026 13:26
@zzylol
zzylol merged commit b18f708 into feat/cost-streaming-linear Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

(dev-tools) Export and visualize planner cost/benefit annotations

1 participant