Skip to content

controller: planner picks sketch family per metric from query intent (5-sketch demo) - #339

Merged
zzylol merged 1 commit into
mainfrom
mvp/planner-pick-sketch-family-per-metric
May 8, 2026
Merged

zzylol merged 1 commit into
mainfrom
mvp/planner-pick-sketch-family-per-metric

Conversation

@zzylol

@zzylol zzylol commented May 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Pin the (sketch, statistic) capability matrix as a first-class module
in controller/src/sketch_algebra/capability_matching.rs and route
bind_workload_typed through it. The MVP demo contract (issue #46)
requires per-metric family selection that the priority-based dispatcher
could not produce — DDSketch (priority 6) always beat KLL (priority 5)
on Quantile, so the rank-err preference for byte-size metrics was
unreachable.

Family-per-statistic mapping (issue #46)

metric family query class
http_requests_total raw passthrough sum / rate / count
http_latency_ms DDSketch quantile (rel-err)
request_size_bytes KLL quantile (rank-err)
unique_users_per_min HLL cardinality
top_endpoint_qps CountSketch top-K
endpoint_request_freq CountMinSketch (CMS) frequency

Capability matrix (final state)

sketch Quantile Cardinality TopK Frequency SumRateCount
DDSketch yes no no no no
KLL yes no no no no
HLL no yes no no no
CountSketch no no yes no no
CMS no no no yes no

SumRateCount has no valid sketch — the agent emits raw OTLP and the
backend computes the answer directly.

sketch_family_override behaviour

The workload-spec's sketch_type_override (= the spec's
sketch_family_override per orchestrator contract) wins over the
capability-matched pick when the override is valid for the statistic
class. An invalid override (HLL on a Quantile workload) is silently
dropped and the planner falls back to the catalog default — the
binding never produces a nonsense (sketch, stat) pair.

Files touched

  • controller/src/sketch_algebra/capability_matching.rs (new) —
    StatisticClass, AccuracyPreference, is_valid_pair,
    pick_family, classify_demo_metric. 14 unit tests pin the matrix.
  • controller/src/sketch_algebra/params.rs — add
    From<SketchType> for SketchKind for the override translation.
  • controller/src/sketch_algebra/mod.rs — register module + re-exports.
  • controller/src/planner/rules.rs::bind_workload_typed — capability-
    matrix-driven family pick; family-specific Bind* rule dispatch
    (bypasses the priority-based dispatcher's tie-break); 10 unit tests
    covering every contract row, override behaviour, and invalid-override
    fallback.

Test plan

  • All 14 capability-matching unit tests pass (cargo test --release sketch_algebra::capability_matching)
  • All 10 planner family-per-metric tests pass (cargo test --release planner::rules::tests::typed_binding_*, *sketch_type_override*, all_six_contract_metrics_produce_expected_family)
  • All 6 contract metrics produce expected SketchExpr family (covered by the all_six_contract_metrics_produce_expected_family sweep test)
  • All 56 sketch_algebra tests pass; pre-existing planner::cost_model::workload_cost_tests::* failures (unresolved Ref("w")) are unrelated to this PR and reproduce on origin/main
  • cargo build --release clean (warnings only)

🤖 Generated with Claude Code

…(5-sketch demo)

Pin the (sketch, statistic) capability matrix as a first-class module
and route `bind_workload_typed` through it. The MVP demo contract
(issue #46) requires per-metric family selection — `http_latency_ms`
must bind to DDSketch (Quantile, rel-err), `request_size_bytes` to KLL
(Quantile, rank-err), `unique_users_per_min` to HLL (Cardinality),
`top_endpoint_qps` to CountSketch (TopK), `endpoint_request_freq` to
CMS (Frequency), and `http_requests_total` to raw passthrough
(SumRateCount). The previous priority-based dispatcher always picked
DDSketch over KLL on Quantile, so the rank-err preference for byte-size
metrics was unreachable.

`sketch_type_override` (the workload-spec's `sketch_family_override`)
wins over the capability-matched default when the override is valid for
the statistic class; an invalid override (HLL on a Quantile workload)
falls back to the catalog default rather than producing a nonsense
binding.

Files:
- `sketch_algebra/capability_matching.rs` (new) — `StatisticClass`,
  `AccuracyPreference`, `is_valid_pair`, `pick_family`,
  `classify_demo_metric`. 14 unit tests pin the matrix.
- `sketch_algebra/params.rs` — add `From<SketchType> for SketchKind`
  for the override translation.
- `planner/rules.rs::bind_workload_typed` — capability-matrix-driven
  pick; family-specific `Bind*` rule dispatch; 10 unit tests covering
  every contract row, override behaviour, and invalid-override
  fallback.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@zzylol
zzylol merged commit 1cec6e2 into main May 8, 2026
zzylol added a commit that referenced this pull request May 8, 2026
Activates the typed-stage-split path for every compose invocation that
includes base.yml, so the demo (and any ad-hoc `docker compose up
controller` against base alone) routes through the typed pipeline:
bootstrap GET (#329), OpAMP-on-connect replan (#334), and per-metric
5-sketch family routing (#339 + #340 + the in-flight stitching PR).

The mvp-multi-stage.yml overlay already sets the same default; this
just promotes the gate to the base layer so it is on without the
overlay.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zzylol added a commit that referenced this pull request May 8, 2026
…_family map (#46) (#342)

PR #339 (planner: per-metric SketchExpr) and PR #340 (emitter:
metric_to_family-driven 5-sketch routing) landed in parallel, but
nothing populated `EdgeStageConfig.metric_to_family`, so the new
routing-connector wire path stayed dormant — every bootstrap and
OpAMP-push emitted single-pipeline DDSketch even with the six
contract metrics declared in `workloads.yaml`.

This stitches them: a new `config::collect_metric_to_family` walks
the workload registry, runs `bind_workload_typed` per metric, and
collects committed sketch families into the HashMap. Metrics that
decline binding (`http_requests_total` raw passthrough,
exact-required, multi-intent) are skipped — the emitter routes
those to `metrics/raw_passthrough` by default.

Wired into both:
  * `main::emit_bootstrap_typed` — also generalised the candidate
    resolution so a registry whose first agent-role entry is raw
    (e.g. `http_requests_total`) walks to the next binding metric
    for the edge_cfg shape rather than falling back to legacy.
  * `replan::Replanner::try_emit_typed_edge_yaml_for_workload` —
    same stitch so OpAMP-pushed YAML on reconnect / replan also
    activates the routing connector.

Acceptance test (`api_tests::bootstrap_emits_5sketch_routing_for_six_contract_metrics`):
loads all six MVP §46 contract metrics into the registry, hits the
bootstrap GET endpoint with `USE_TYPED_STAGE_SPLIT=1`, and asserts
the emitted YAML contains all 5 sketch processors, `routing` under
`connectors:` (not `processors:`), all 6 named pipelines, and a
`route() where metric.name == "..."` rule for each sketched metric.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@zzylol
zzylol deleted the mvp/planner-pick-sketch-family-per-metric branch May 9, 2026 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant