Skip to content

feat(cost): cost complete streaming summary DAGs - #335

Merged
zzylol merged 9 commits into
mainfrom
feat/cost-streaming-summary-dag
Sep 4, 2026
Merged

zzylol merged 9 commits into
mainfrom
feat/cost-streaming-summary-dag

Conversation

@zzylol

@zzylol zzylol commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Why

Summing per-summary estimates can miss merge, subtract, delete, join, readout, shared-state, and retained-input work in a nested SummaryExpr DAG.

What

Estimate the complete selected summary DAG and reject incomplete or contradictory physical evidence.

How

  • Traverse the whole SummaryExpr DAG recursively.
  • Deduplicate shared work by provider-owned physical identity.
  • Validate every edge, source lineage, operator cost, lifecycle schedule, and retained state.
  • Compute CPU and I/O once per physical operation and peak memory from liveness.

Before this PR

Lifecycle costing could rely on partial per-state totals and omit parent or shared operations.

After this PR

A candidate is available only when its complete streaming SummaryExpr DAG has consistent evidence.

Stack

Depends on #334. The next PR integrates complete estimates into global planner selection.

Module boundary

Summary-maintenance evidence is separated from its planner adapter. Raw streaming baselines now use the shared EvidenceBackedPhysicalDag; the duplicate streaming-only physical DAG/statistics container has been removed.

@zzylol
zzylol force-pushed the feat/cost-streaming-lifecycle-ranking branch from 631ac15 to f88029c Compare September 3, 2026 16:29
@zzylol
zzylol force-pushed the feat/cost-streaming-summary-dag branch from 0f77c4e to 071c2a6 Compare September 3, 2026 16:29
@zzylol
zzylol force-pushed the feat/cost-streaming-lifecycle-ranking branch from f88029c to 77658a1 Compare September 3, 2026 16:45
@zzylol
zzylol force-pushed the feat/cost-streaming-summary-dag branch 2 times, most recently from 8e30f91 to 555113c Compare September 3, 2026 16:50
@zzylol
zzylol force-pushed the feat/cost-streaming-lifecycle-ranking branch from 089c8c1 to 40db2f1 Compare September 4, 2026 00:12
@zzylol
zzylol force-pushed the feat/cost-streaming-summary-dag branch from 1f80380 to 0e9f092 Compare September 4, 2026 00:35
@zzylol
zzylol force-pushed the feat/cost-streaming-lifecycle-ranking branch from 40db2f1 to 0219280 Compare September 4, 2026 02:25
@zzylol
zzylol force-pushed the feat/cost-streaming-summary-dag branch from 0e9f092 to d02bf12 Compare September 4, 2026 02:26
@zzylol
zzylol force-pushed the feat/cost-streaming-lifecycle-ranking branch from 0219280 to d72457a Compare September 4, 2026 02:50
@zzylol
zzylol force-pushed the feat/cost-streaming-summary-dag branch from d02bf12 to 5f71dee Compare September 4, 2026 03:00
@zzylol
zzylol force-pushed the feat/cost-streaming-lifecycle-ranking branch from 48c15ca to 6e60fab Compare September 4, 2026 03:26
@zzylol
zzylol force-pushed the feat/cost-streaming-summary-dag branch from 5f71dee to 4cd0ee2 Compare September 4, 2026 03:29
@zzylol
zzylol force-pushed the feat/cost-streaming-lifecycle-ranking branch from 6e60fab to b4c5480 Compare September 4, 2026 04:01
@zzylol
zzylol force-pushed the feat/cost-streaming-summary-dag branch from 4cd0ee2 to 673e74a Compare September 4, 2026 04:01
@zzylol
zzylol changed the base branch from feat/cost-streaming-lifecycle-ranking to main September 4, 2026 11:27
@zzylol
zzylol merged commit 8909ea1 into main Sep 4, 2026
3 checks passed
@zzylol
zzylol deleted the feat/cost-streaming-summary-dag branch September 7, 2026 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant