docs(sketch): L4 — planning-time vs. serving-time design writeup - #156
Merged
zzylol merged 6 commits intoJul 22, 2026
Merged
Conversation
Explains the split ASAPController#155 (SummaryExecutor/execute) adds: planning-time L4 (asap_plan::bind, QueryExpr -> L4Node, a decision made symbolically once per query shape) vs. serving-time L4 (this module, L4Node -> Value, a lookup done on every query against whatever's actually materialized). Covers the trait split of responsibility, the nested-composition rules, what's out of scope, and the planned data_plane consumer. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…uplicating it The trait signature, nested-composition rules, and out-of-scope notes were already verbatim in crates/sketch/src/exec.rs's module doc comments -- this doc now just frames planning-time vs. serving-time L4 and points there instead of keeping a second copy in sync. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ead of duplicating it" This reverts commit d434e20.
…ecution-model.md design.md's "asap-sketch -- Layer 4 IR" section carried the SketchExpr struct definition (the pre-shipped placeholder name), the per-node schema table, and the two type-system invariants -- all planning-time L4 content that belongs with the rest of the L4 story now that l4node-execution-model.md exists. Moves it there, rewritten against the actual shipped names (SummaryExpr/L4Node/L4Schema/L4DataType, real crates/sketch/src/expr.rs field shapes) instead of the old SketchExpr/ SketchKind/GroupKey placeholders and their translation footnote. design.md keeps a short pointer plus the two Source/data-model notes that aren't really about the IR shape itself. Doesn't chase every other SketchExpr mention elsewhere in design.md (the layer table, the glossary, the worked example) -- scoped to the one section that duplicated the actual struct definitions; broader terminology consistency is a separate cleanup. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
zzylol
force-pushed
the
docs/l4node-execution-model
branch
from
July 22, 2026 17:55
c840518 to
48780ab
Compare
SummaryMerge/SummaryJoin/SummarySubtract/SummaryDelete are real types exec.rs (mostly) knows how to run, but nothing produces them yet -- worth being explicit about which half of "defined" vs "actually happens" each is in, rather than leaving it implicit across the doc. Same for SummaryKind: all 14 variants are wired into boundary::implementation_for, split into DefaultCostModel's pick vs. what only a custom CostModel reaches. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
Author
|
@milindsrivastava1997 I will merge this, and solve the doc restructure issue in #157 for all 5 layers. |
zzylol
added a commit
that referenced
this pull request
Jul 22, 2026
Same move as l4node-execution-model.md (#155/#156) for L4: each layer gets its own doc instead of living as a subsection of the 1500+ line design.md, which design.md now links to instead of duplicating. - l1-query-language.md, l2-logical-plan.md: short, these layers barely have design content beyond "here's the parser." - l3-intent-algebra.md: the real content -- QueryExpr/AggIntent design rules, schema flow, the three-metadata-sources table. Rewritten against a pointer to the real source files instead of reproducing the full struct definitions inline, since the old inline copy here had already drifted stale (stale AggIntent::TopK/Rate/Increase field shapes, same kind of drift l4node-execution-model.md's L4 section had before this round of cleanup) -- points at crates/ir/src/intent_algebra/*.rs as the source of truth instead of re-copying it, so it can't drift the same way again. - l5-physical-plan.md: moved close to verbatim -- already correctly labeled "planned, not yet built" from an earlier reconciliation pass, no staleness to fix, just relocated. design.md keeps a two-line pointer per layer. Section 6's intro no longer claims "(layers 1-3 + driver)" now that the layer IRs themselves live elsewhere -- points to all five layer docs instead. Doesn't chase every remaining cross-reference to the old section numbers/content scattered elsewhere in design.md (glossary, worked examples, open-questions) -- those still resolve (the §6 subsection headers stay, now as pointers), just not rewritten in place. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
zzylol
added a commit
that referenced
this pull request
Jul 22, 2026
…de (#155) * feat(sketch): SummaryExecutor -- serving-time execution model for L4Node asap-sketch defines the L4 tree shape and asap-plan defines how to build one from L3, but nothing said what it means to *run* one against already-materialized state at query time -- every deployment answering queries would otherwise reinvent the same recursive walk and merge preconditions independently. Adds SummaryExecutor (the trait a deployment implements to plug in its own storage/lookup/sketch-math/readout) and execute() (the generic recursive walk over L4Node that calls into it). asap-sketch owns the structural rules -- which nestings are valid, that a SummaryMerge's children must agree on (SummaryKind, SummaryParams), propagated through arbitrary nesting depth via ExecOutcome::State rather than re-derived per node. The deployment owns everything requiring actual sketch math (this crate has none) or actual storage. 9 tests cover single-leaf readout, multi-candidate merge, nested SummaryAgg (quantile-of-sum shape), merge-of-merges, and the three structural error cases (empty merge, a Value-producing merge child, mismatched kind/params across merge children). Design writeup in a stacked follow-up PR (docs/l4node-execution-model.md); this module's own doc comments are the authoritative detail. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * docs(sketch): trim exec.rs's comments -- covered by l4node-execution-model.md The design rationale (planning vs. serving L4, nested-composition rules, why each error variant exists) now lives in the stacked doc PR, not duplicated in-line. Keeps short, standard API docs; points to docs/l4node-execution-model.md for the reasoning instead of restating it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * docs(sketch): L4 — planning-time vs. serving-time design writeup (#156) * docs(sketch): L4 -- planning-time vs. serving-time design writeup Explains the split ASAPController#155 (SummaryExecutor/execute) adds: planning-time L4 (asap_plan::bind, QueryExpr -> L4Node, a decision made symbolically once per query shape) vs. serving-time L4 (this module, L4Node -> Value, a lookup done on every query against whatever's actually materialized). Covers the trait split of responsibility, the nested-composition rules, what's out of scope, and the planned data_plane consumer. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * docs(sketch): point l4node-execution-model.md at exec.rs instead of duplicating it The trait signature, nested-composition rules, and out-of-scope notes were already verbatim in crates/sketch/src/exec.rs's module doc comments -- this doc now just frames planning-time vs. serving-time L4 and points there instead of keeping a second copy in sync. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * Revert "docs(sketch): point l4node-execution-model.md at exec.rs instead of duplicating it" This reverts commit d434e20. * docs: move the L4 IR struct definitions from design.md into l4node-execution-model.md design.md's "asap-sketch -- Layer 4 IR" section carried the SketchExpr struct definition (the pre-shipped placeholder name), the per-node schema table, and the two type-system invariants -- all planning-time L4 content that belongs with the rest of the L4 story now that l4node-execution-model.md exists. Moves it there, rewritten against the actual shipped names (SummaryExpr/L4Node/L4Schema/L4DataType, real crates/sketch/src/expr.rs field shapes) instead of the old SketchExpr/ SketchKind/GroupKey placeholders and their translation footnote. design.md keeps a short pointer plus the two Source/data-model notes that aren't really about the IR shape itself. Doesn't chase every other SketchExpr mention elsewhere in design.md (the layer table, the glossary, the worked example) -- scoped to the one section that duplicated the actual struct definitions; broader terminology consistency is a separate cleanup. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * docs(sketch): list currently-supported L4 operators and SummaryKinds SummaryMerge/SummaryJoin/SummarySubtract/SummaryDelete are real types exec.rs (mostly) knows how to run, but nothing produces them yet -- worth being explicit about which half of "defined" vs "actually happens" each is in, rather than leaving it implicit across the doc. Same for SummaryKind: all 14 variants are wired into boundary::implementation_for, split into DefaultCostModel's pick vs. what only a custom CostModel reaches. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * docs(sketch): rename section to "Supported operators" Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
zzylol
added a commit
that referenced
this pull request
Jul 22, 2026
Same move as l4node-execution-model.md (#155/#156) for L4: each layer gets its own doc instead of living as a subsection of the 1500+ line design.md, which design.md now links to instead of duplicating. - l1-query-language.md, l2-logical-plan.md: short, these layers barely have design content beyond "here's the parser." - l3-intent-algebra.md: the real content -- QueryExpr/AggIntent design rules, schema flow, the three-metadata-sources table. Rewritten against a pointer to the real source files instead of reproducing the full struct definitions inline, since the old inline copy here had already drifted stale (stale AggIntent::TopK/Rate/Increase field shapes, same kind of drift l4node-execution-model.md's L4 section had before this round of cleanup) -- points at crates/ir/src/intent_algebra/*.rs as the source of truth instead of re-copying it, so it can't drift the same way again. - l5-physical-plan.md: moved close to verbatim -- already correctly labeled "planned, not yet built" from an earlier reconciliation pass, no staleness to fix, just relocated. design.md keeps a two-line pointer per layer. Section 6's intro no longer claims "(layers 1-3 + driver)" now that the layer IRs themselves live elsewhere -- points to all five layer docs instead. Doesn't chase every remaining cross-reference to the old section numbers/content scattered elsewhere in design.md (glossary, worked examples, open-questions) -- those still resolve (the §6 subsection headers stay, now as pointers), just not rewritten in place. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
milindsrivastava1997
approved these changes
Jul 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked on #155. Design writeup for the split #155 adds:
asap_plan::bind) —QueryExpr -> L4Node, a decision made symbolically once per query shape, no reference to what's actually stored anywhere. The half that already existed (docs/design.md's "Sketch binding is already committed by L4").crates/sketch/src/exec.rs, feat(sketch): SummaryExecutor — serving-time execution model for L4Node #155) —L4Node -> Value, a lookup done on every query against whatever's actually materialized. The half that didn't have a home until feat(sketch): SummaryExecutor — serving-time execution model for L4Node #155.Covers the
SummaryExecutortrait's split of responsibility, the nested-composition rules (what can be aSummaryMergechild, merge-of-merges transitivity, the deliberately-openSummaryAgg-over-SummaryEstimatequestion), what's out of scope (L5 placement,SummaryJoin/SummarySubtract/SummaryDelete), and the planneddata_planeconsumer.Test plan
N/A — docs only, no code changes (module doc comments already changed in #155).
🤖 Generated with Claude Code