docs: define ASAPPlanner input, output, and workflow by zzylol · Pull Request #445 · ProjectASAP/ASAPPlanner

zzylol · 2026-09-18T18:38:54Z

Why

Issue #438 shows that the current documentation exposes individual planner stages without first defining the supported end-to-end integration workflow. Callers cannot easily tell which inputs are required, when lifecycle planning is necessary, or whether PlanSpace, a ranked group, or a materialized DAG is the final output. Related terminology and API-surface ambiguity is tracked in #427, and #428.

Before this PR

A caller could discover search_workload*, cost_sorted, global_selection, and lifecycle APIs independently and reasonably conclude that each was a separate valid end state. For example, a recurring summary could be structurally selected without making clear that this does not establish that maintaining it is cheaper than exact recomputation. Window semantics, summary frameworks, physical layouts, window edges, and physical handoffs were also easy to conflate.

After this PR

The design documentation starts from one outer workflow: canonical roots and requirements enter ASAPPlanner, and PlanSpace is the canonical logical output. It then explicitly documents:

required, optional, and optimization-specific inputs;
the meaning and categories of evidence;
PlanSpace, ranked views, materialized DAGs, and lifecycle-aware outputs;
inspection, structural-selection, and lifecycle-aware workflows and their promises;
replanning as a fresh invocation with a new workload/evidence snapshot; and
ownership-specific vocabulary for query windows, summary frameworks, physical layouts, pane layouts, window edges, physical handoffs, and comparison scopes.

The lifecycle-aware workflow is identified as the recommended path before claiming that maintained summary state is preferable to raw execution. Physical binding, deployment, transitions, and execution remain downstream responsibilities. The document also states that public Rust visibility does not automatically make a type part of the recommended integration surface.

Validation

git diff --check
verified all relative links added by the document resolve locally

Closes #438
Related to #427 and #428
Related to ProjectASAP/ASAPQuery-backend#734

milindsrivastava1997 · 2026-09-18T21:13:46Z

+}
+```
+
+| Field | Required | Purpose |


It may be helpful to add a column on why each field is important. Simple example and intuition will help. For e.g. I do not yet get how predictability is useful or why I must provide it.

milindsrivastava1997 · 2026-09-18T21:14:42Z

+`PlanSpace` is Planner's canonical output. It contains:
+
+* canonical workload roots;
+* memo groups for discovered target sub-DAGs;


Is "memo" short for memoization? Should change this name

milindsrivastava1997 · 2026-09-18T21:15:24Z

+
+Evidence is input to planning. The current library does not collect every kind
+in one `PlanningWorkload` field; each fact enters through the interface that
+consumes it:


"each fact enters through the interface that
consumes it:" -- remove. this means nothing

milindsrivastava1997 · 2026-09-18T21:16:39Z

+`PlanSpace::cost_sorted` provides a ranked view for inspection:
+
+```text
+Vec<RankedGroup {


What is target? Is this different from root? Also, pls define RankedGroup conceptually

milindsrivastava1997 · 2026-09-18T21:17:46Z

+PlanSpace
+    |
+    v
+global_selection


It would be helpful to link these to other docs where each step here or function may be described in more detail. I think you did have docs for that(?)

milindsrivastava1997 · 2026-09-18T21:20:00Z

+and update rate are read from the optional `DataWorkload`. Missing required
+facts remain unknown rather than being treated as zero.
+
+`SummaryMaintenanceLifecyclePlan` additionally records:


Is this the output of "lifecycle-aware planning"? If so, pls mention explicitly
Also, what what is the public helper function name?

milindsrivastava1997 · 2026-09-18T21:21:01Z

+
+It is still **not an executable deployment plan**. Physical operator binding, placement, storage, and execution remain downstream responsibilities.
+
+### Lifecycle-aware helper


What is the relationship between lifecycle-aware planning and "selection and materialization helper"? Can I use one but not the other? If I use both, is there a specific order to them?

milindsrivastava1997 · 2026-09-18T21:21:15Z

+
+### Selection and materialization helper
+
+`PlanSpace::global_selection*` coordinates decisions across memo groups.


Not sure what this means

Also, not sure what materialize is doing

milindsrivastava1997 · 2026-09-18T21:21:49Z

+
+This view is useful for debugging, explanation, or downstream optimization. Candidate presence does not imply physical deployability.
+
+### Selection and materialization helper


Is there a single function for this? It's ok if not, but just mention that this is one function call to a public API, or multiple function calls.

milindsrivastava1997 · 2026-09-18T21:23:21Z

+PlanSpace
+    -> global_selection
+    -> materialize
+    -> selected Post-ASAP DAGs


In the "Selection and materialization helper", the output is a single DAG but here it is multiple.

Selvomega · 2026-09-19T15:41:08Z

+ASAPPlanner is a **logical planning library**. Its input is a planning workload
+plus the models, evidence, and deployment capabilities needed by the requested
+planning workflow. Its canonical output is a `PlanSpace` containing the legal
+Post-ASAP alternatives for the workload.


This is much clearer, thanks!

Selvomega · 2026-09-19T15:49:36Z

+| Input | Fields | Required |
+|---|---|---:|
+| `PlanningWorkload.query_workload` | Query language and one-time/repeating query workloads | Yes |
+| `PlanningWorkload.data_workload` | Data arrival and optional evidence about ingestion, cardinality, and distribution | Conditional: required for PromQL; otherwise optional |


What is the default setting of this field?

Selvomega · 2026-09-19T15:50:19Z

+planning workflow. Its canonical output is a `PlanSpace` containing the legal
+Post-ASAP alternatives for the workload.
+
+### Input fields at a glance


As far as I know, data catalog is also required. In the following table, which item contains data catalog?

Selvomega · 2026-09-19T16:05:29Z

+[Ranking](#ranked-view), [selection and
+materialization](#selection-and-materialization-helper), and
+[lifecycle](#lifecycle-aware-helper) APIs are views or helper operations over
+this output, not additional top-level Planner outputs.


Hmm this is very interesting...
Sounds like, ASAPPlanner is providing a PlanSpace as output, along with some API functions to operate over it.
As I perceived, this implies that the output PlanSpace itself does not have any notion of lifecycle, which will only appear after you apply the lifecycle related APIs.
This somehow goes back to the question that how should users use these APIs. Is it proper for users to use it without the lifecycle API calls? If not, then why not design DAGs with the notion of lifecycle as the output?

Selvomega · 2026-09-19T16:14:59Z

+These inputs are supplied alongside `PlanningWorkload`, rather than nested
+inside it:


Cool now I see where the data catalog is. But why didn't you put it into the Input table at the beginning?

Selvomega · 2026-09-19T17:03:41Z

+and update rate are read from the optional `DataWorkload`. Missing required
+facts remain unknown rather than being treated as zero.
+
+`SummaryMaintenanceLifecyclePlan` additionally records:


What is the relation between this SummaryMaintenanceLifecyclePlan and post-ASAP-DAG? Does the former include the latter? Or they are paralleled concept?

Selvomega · 2026-09-19T17:04:23Z

+
+### Lifecycle-aware helper
+
+Lifecycle-aware planning is an optional operation on an existing `PlanSpace`.


Why it is optional? In what scenario it is required?

Selvomega · 2026-09-19T17:07:56Z

+
+It is still **not an executable deployment plan**. Physical operator binding, placement, storage, and execution remain downstream responsibilities.
+
+### Lifecycle-aware helper


Can the helper here also select and materialize to an optimized plan? Or it must be combined with the selection and materialization helper?

Selvomega · 2026-09-19T17:11:07Z

+
+## Output
+
+### `PlanSpace<Id>`


Actually I still don't understand why the optimal plan selection logic is not placed inside the input-to-output black box, and why not just expose one optimal plan as output.

Selvomega · 2026-09-19T17:44:38Z

+
+---
+
+## Workflows


This is clear, thanks!

However, according to my understanding, I feel the output of ASAPPlanner should not be a PlanSpace, but should be a specific optimal plan, which can have / not have notion of lifecycle.
Since the selection and materialization seems to be applied to all downstream usages of the PlanSpace, and there is indeed no reason for user to see a space of candidates.

Another thing I feel important: We should have the logical picture and implementation separated. For example, in my opinion, I think "single optimal DAG w/ or w/o notion of lifecycle" is a better logical output, instead of the current PlanSpace. I admit there currently seems to be a gap between this logical picture and code implementation, but imo we should first get the logical picture clear and straight, and use that as a guide to fix the code.

Selvomega · 2026-09-19T17:52:44Z

+### Output at a glance
+
+| Output | Fields or contents | Meaning |
+|---|---|---|
+| `PlanSpace<Id>` | The legal candidate Post-ASAP DAGs for the workload, represented compactly as canonical roots, memoized alternatives, and cross-group composition information | The ASAPPlanner output |
+
+[Ranking](#ranked-view), [selection and
+materialization](#selection-and-materialization-helper), and
+[lifecycle](#lifecycle-aware-helper) APIs are views or helper operations over
+this output, not additional top-level Planner outputs.


Just realized that output described here is inconsistent with what is written in the docs/design_docs/architecture/README.md diagram. This is not good, and please double check!

docs: define planner input output contract

e71a6c4

zzylol requested a review from milindsrivastava1997 September 18, 2026 18:38

docs: clarify planner and runtime vocabulary

b4841b3

zzylol changed the title ~~docs: define ASAPPlanner input, output, and workflows~~ docs: define ASAPPlanner input, output, and workflow Sep 18, 2026

zzylol and others added 4 commits September 18, 2026 14:44

Update input-output-workflow.md

c4cae3d

Update input-output-workflow.md

f83963e

docs: describe planning workload fields

a045645

Update input-output-workflow.md

5173532

zzylol removed the request for review from milindsrivastava1997 September 18, 2026 18:54

zzylol and others added 16 commits September 18, 2026 18:56

docs: summarize planner input and output fields

9e74b15

docs: keep overview at workload boundary

8ecb668

docs: define one planner output

0a8c001

docs: name frontend-specific dependencies

26bca3d

docs: explain compact plan space representation

ddedb2a

docs: move plan space details after overview

449c56f

Update input-output-workflow.md

80b48cf

docs: mirror workload nesting in input sections

99a64e3

docs: link overview to output helpers

f7c50b7

docs: explain caller-owned plan root IDs

4983f5b

docs: keep inputs at end-to-end boundary

eba1174

docs: move lifecycle parameters to helper

afb9a68

docs: classify evidence as planner input

f11d3bf

docs: mark replanning as future work

3995bef

docs: state when planning evidence is required

03c712e

docs: separate lifecycle helper parameters

235eb6a

zzylol requested review from Selvomega and milindsrivastava1997 September 18, 2026 20:06

zzylol mentioned this pull request Sep 18, 2026

docs: define backend plan split and SDS state contract ProjectASAP/ASAPQuery-backend#737

Open

milindsrivastava1997 reviewed Sep 18, 2026

View reviewed changes

Selvomega requested changes Sep 19, 2026

View reviewed changes

Selvomega reviewed Sep 19, 2026

View reviewed changes

Selvomega mentioned this pull request Sep 19, 2026

[Design] Materializing the Logical Input / Output and Workflow Picture Onto lib Functions Visible to Users #453

Open


		It is still not an executable deployment plan. Physical operator binding, placement, storage, and execution remain downstream responsibilities.

		### Lifecycle-aware helper


		### Selection and materialization helper

		`PlanSpace::global_selection*` coordinates decisions across memo groups.


		This view is useful for debugging, explanation, or downstream optimization. Candidate presence does not imply physical deployability.

		### Selection and materialization helper

		These inputs are supplied alongside `PlanningWorkload`, rather than nested
		inside it:


		### Lifecycle-aware helper

		Lifecycle-aware planning is an optional operation on an existing `PlanSpace`.


		---

		## Workflows

Conversation

zzylol commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

Before this PR

After this PR

Validation

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Selvomega Sep 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

zzylol commented Sep 18, 2026 •

edited

Loading

Selvomega Sep 19, 2026 •

edited

Loading