Skip to content

feat(controller): Phase F — CostModel workload_cost + Schema unique_keys CSE legality - #277

Merged
zzylol merged 1 commit into
mainfrom
feat/controller-phase-f-cost-model-and-schema-cse
May 6, 2026
Merged

zzylol merged 1 commit into
mainfrom
feat/controller-phase-f-cost-model-and-schema-cse

Conversation

@zzylol

@zzylol zzylol commented May 6, 2026

Copy link
Copy Markdown
Contributor

Summary

Per design.md §6 core::cost (line ~997) and Schema flow (line ~425) — the workload-level cost-model entry point + the load-bearing consumer of Schema::unique_keys. Phase A/B/D shipped the typed types, the L3 IR DAG with Schema::unique_keys, and the language wrap; Phase F is what makes unique_keys "load-bearing" and what credits shared sub-DAGs in the bundled cost (the proof point that the design's reuse-aware planning is reachable from the existing IR).

  • planner::cost_model::workload_cost — walks the L3 IR DAG (intent_algebra::QueryExpr) post-order, memoises by LetBinding name, credits each shared producer once across consumers. Returns WorkloadCost { total_dollars, per_root_breakdown, reused_savings }. The reused_savings field exposes the gap between the bundled total and the naive sum-over-roots (design.md §6 line ~1023).
  • intent_algebra::schema::cse_reuse_is_legal — gatekeeper. Two QueryExpr::Ref consumers may share a producer only when the producer's output schema has at least one unique_keys set and the consumer count is ≥ 2. The proof point that Schema::unique_keys is load-bearing per design.md §6 line ~1356.
  • intent_algebra::cse::dedupe_subtrees — basic implementation of the workload-level CSE pass. Detects shared Aggregate children across ≥2 roots, gates on cse_reuse_is_legal, hoists into a LetBinding. Full alpha-equivalence + nested-CSE algorithm is downstream.

design.md grows two "Implementation status" subsections (one under core::cost, one under the Schema flow table) describing what shipped vs. what's deferred (per-plan latency split, ReusedComponent enumeration, schema-equivalent-but-not-identical sub-tree merging).

Files touched

Strictly within the spec's allow-list:

  • controller/src/planner/cost_model.rs — extended with workload_cost, WorkloadCost, WorkloadCostPlan, per-node cost primitives, 7 tests.
  • controller/src/intent_algebra/schema.rs — extended with cse_reuse_is_legal, CseError, 4 new tests.
  • controller/src/intent_algebra/cse.rsnew. dedupe_subtrees, CseWorkloadPlan, 4 tests.
  • controller/src/intent_algebra/mod.rsmod cse; + re-exports for the new public surface.
  • controller/docs/design.md — Implementation-status subsections under core::cost and Schema flow; Phase F entry in the Phase B implementation-status section.

Untouched (per spec): sketch_algebra/, intent_algebra/{agg_intent,query_expr,lower}.rs, types_v2.rs, query_language/, language_logical_plan/, planner/{rules,stage_split}.rs, analyzer.rs, replan.rs, opamp/, query_parser/, types.rs, config/*, anything outside controller/.

Test plan

15 unit tests added:

  • planner::cost_model::workload_cost_tests (7) — single-root degenerate, two-roots-no-sharing, two-roots-shared-window-credits-once (the design.md batched-queries example), three-roots-two-share-partial, three-roots-all-share, unused-binding-zero-savings, unresolved-ref-errors.
  • intent_algebra::schema::tests (4 new) — cse_reuse_legal_when_unique_keys_set, cse_reuse_illegal_when_unique_keys_empty, cse_reuse_rejects_single_consumer, cse_reuse_consumer_check_precedes_unique_key_check.
  • intent_algebra::cse::tests (4) — dedupe_subtrees_empty_input, dedupe_subtrees_single_root_passthrough, dedupe_subtrees_basic (design.md batched-queries example), dedupe_subtrees_no_shared_subexpr.

Verification

  • cargo build --release -p controller — clean (132 warnings, same as origin/main).
  • cargo clippy --release --bin controller -- -D warnings — same 162 errors as origin/main baseline; Phase F adds zero new clippy errors. (All 162 are pre-existing in unmodified files: algebra/directory.rs, types.rs, etc.)
  • No docker activity. Sweep on PID 2862786 untouched.
  • No submodule pointer drift.

Note on cargo test status

The controller's cargo test target was already broken at origin/main (post-#275): nine data_sink field-init errors in src/config/agent.rs, src/config/asapquery_backend.rs, src/config/precompute.rs, and src/main.rs test code. This is the pre-existing baseline blocker explicitly called out in the orchestrator spec (controller/src/config/* — pre-existing test baseline blocker, separate fix in flight #28). Phase F's tests are written and reviewed for logical correctness; they will run green once #28 unblocks the test target.

cargo build --release and cargo clippy --release --bin controller are clean of any new Phase F warnings or errors — confirmed by stashing the diff and rerunning baseline (same 132 build warnings / 162 clippy errors).

🤖 Generated with Claude Code

…eys CSE legality

Per design.md §6 `core::cost` (line ~997) and Schema flow (line ~425),
land the workload-level cost-model entry point + the load-bearing
consumer of `Schema::unique_keys`. Phase A/B/D shipped the typed types,
the L3 IR DAG with `Schema::unique_keys`, and the language wrap; Phase F
is what makes `unique_keys` "load-bearing" and what credits shared
sub-DAGs in the bundled cost.

What lands:

- `planner::cost_model::workload_cost(plan: &WorkloadCostPlan<'_>) ->
  Result<WorkloadCost, QueryExprError>` — walks the L3 IR DAG (Phase B's
  `intent_algebra::QueryExpr`) post-order, memoises by `LetBinding`
  name, credits each shared producer once across consumers. Returns
  `WorkloadCost { total_dollars, per_root_breakdown, reused_savings }`.
  `reused_savings` exposes the gap between the bundled total and the
  naive sum-over-roots, for EXPLAIN / observability per design.md §6
  line ~1023.
- `planner::cost_model::WorkloadCostPlan { bindings, roots }` — the
  cost-model's view of `types_v2::WorkloadPlan` carrying real
  `&QueryExpr` references rather than the `QueryExprPlaceholder` JSON-
  wire string. Collapses into `types_v2::WorkloadPlan` when the
  placeholder is swapped for live `QueryExpr` downstream.
- `intent_algebra::schema::cse_reuse_is_legal(producer_schema,
  consumer_count) -> Result<(), CseError>` — the gatekeeper. Two
  `QueryExpr::Ref` consumers may share a producer only when the
  producer's output schema has at least one `unique_keys` set and the
  consumer count is ≥ 2. This is the proof point that `Schema::unique_keys`
  is load-bearing — without it the deduper conservatively refuses to
  share and reuse "drops on the floor" (design.md §6 line ~1356).
- `intent_algebra::cse::dedupe_subtrees(roots) -> CseWorkloadPlan` —
  basic implementation of the workload-level CSE pass. Detects shared
  `Aggregate` children across ≥2 roots, gates on `cse_reuse_is_legal`,
  hoists into a `LetBinding`. The full alpha-equivalence + nested-CSE
  algorithm is downstream — Phase F lands the gate + the basic case so
  the cost-model side has something to credit.

Per-node cost primitives at L3 (`node_cost_scan`, `node_cost_window`,
`node_cost_aggregate`, `intent_cost`) are coarse-but-monotonic
placeholders calibrated against schema width and intent kind. What
Phase F pins is the *shape* (positive, additive, savings invariant
`bundled_total ≤ naive_sum`); calibration against real benchmarks is
downstream.

Tests added (15):

- `planner::cost_model::workload_cost_tests` (7):
  - `workload_cost_single_root_equals_query_cost` — degenerate case
  - `workload_cost_two_roots_no_sharing_equals_sum` — independent queries
  - `workload_cost_two_roots_shared_window_credits_once` — design.md
    batched-queries example: 2 quantile queries share Window+Scan;
    `reused_savings ≈ shared_cost`
  - `workload_cost_three_roots_two_share_partial` — q1+q2 share, q3
    independent; savings = 1× shared_cost
  - `workload_cost_three_roots_all_share_one_binding` — three consumers,
    savings = 2× shared_cost
  - `workload_cost_unused_binding_is_zero_savings_not_negative` —
    defensive non-negative invariant
  - `workload_cost_unresolved_ref_errors` — `Ref` to undeclared name
    surfaces as `UnresolvedRef`
- `intent_algebra::schema::tests` (4 added — total 8 with the 4
  pre-existing schema tests):
  - `cse_reuse_legal_when_unique_keys_set` — green path
  - `cse_reuse_illegal_when_unique_keys_empty` — refused without
    unique_keys
  - `cse_reuse_rejects_single_consumer` — short-circuit for count < 2
  - `cse_reuse_consumer_check_precedes_unique_key_check` — error-
    ordering invariant
- `intent_algebra::cse::tests` (4):
  - `dedupe_subtrees_empty_input`
  - `dedupe_subtrees_single_root_passthrough`
  - `dedupe_subtrees_basic` — design.md batched-queries example: two
    queries with identical `Window` sub-trees get hoisted
  - `dedupe_subtrees_no_shared_subexpr` — no fan-in detected → no
    binding emitted

design.md grows two "Implementation status" subsections — one under
`core::cost` describing what `workload_cost` ships and what's deferred
(per-plan latency split, `ReusedComponent` enumeration), and one under
the Schema flow table describing `cse_reuse_is_legal` + the basic
`dedupe_subtrees` shape.

Note on cargo test status. The controller's `cargo test` target was
already broken at origin/main (post-#275): nine `data_sink` field-init
errors in `src/config/agent.rs`, `src/config/asapquery_backend.rs`,
`src/config/precompute.rs`, and `src/main.rs` test code. This is the
pre-existing baseline blocker called out in the orchestrator spec
(separate fix in flight, #28). `cargo build --release -p controller`
and `cargo clippy --release --bin controller` both come out at the same
warning / error count as `origin/main` (132 build warnings, 162 clippy
errors — all pre-existing in unmodified files). Phase F adds zero new
warnings or clippy errors.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@zzylol
zzylol merged commit 889b2ac into main May 6, 2026
@zzylol
zzylol deleted the feat/controller-phase-f-cost-model-and-schema-cse branch May 9, 2026 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant