Skip to content

fix: planner binds all 6 contract metrics into routing table (HLL/CountSketch/CountMin) - #345

Merged
zzylol merged 1 commit into
mainfrom
fix/stitching-binds-hll-cs-cms
May 8, 2026
Merged

zzylol merged 1 commit into
mainfrom
fix/stitching-binds-hll-cs-cms

Conversation

@zzylol

@zzylol zzylol commented May 8, 2026

Copy link
Copy Markdown
Contributor

Summary

The 5-sketch routing-connector wire YAML emitted by the bootstrap GET
(/api/v1/collector-config/agent) was missing 3 of the 6 MVP §46 contract
metrics. Symptom from the live demo:

http_requests_total_latency_ms → ddsketch_path        ✅
request_size_bytes              → kll_path             ✅
http_freshness_probe_warm       → raw_passthrough      ✅ (demo plumbing)
http_freshness_probe_archive    → raw_passthrough      ✅ (demo plumbing)
unique_users_per_min            → should hit hll_path           ❌
top_endpoint_qps                → should hit countsketch_path   ❌
endpoint_request_freq           → should hit countminsketch_path ❌

The workload registry loaded all 8 entries (count=8), but
collect_metric_to_family walked them and only emitted 2 sketched routes.

Root cause — two missing stitches

  1. WorkloadEntry silently ignored sketch_family_override. The struct in
    controller/src/config/workloads.rs had no field for it; serde defaults
    to ignoring unknown fields, so the operator's sketch_family_override: HLL / CountSketch / CountMinSketch round-tripped to None. The
    pre-pop loop in main() then passed QuerySpec.sketch_type = None to
    the analyzer.

  2. bind_workload_typed early-returned on w.exact_required = true at
    controller/src/planner/rules.rs:81. The PromQL parser sets that flag
    when the inner walk synthesises an AggFunc::Sum over a bare
    VectorSelector — which happens for count(metric),
    topk(K, metric), and rate(metric[5m]) (the query_strings on
    workload entries 6, 7, 8). The metric-name match in
    classify_demo_metric and the sketch_type_override field never got
    consulted because the early return happened first.

    Carved out an exception: when the metric is a contract row
    (classify_demo_metric returns Some) or carries an explicit override,
    those signals win over exact_required because both are explicit
    "sketch this metric" instructions.

Files

  • controller/src/config/workloads.rs — add sketch_family_override
    (case-insensitive SketchType deserialiser) and target_path fields
    to WorkloadEntry; new WorkloadRegistry::from_entries for testing.
  • controller/src/main.rs — thread entry.sketch_family_override into
    QuerySpec.sketch_type in the registry pre-pop loop (lines ~234).
  • controller/src/planner/rules.rs:81-99 — guard the exact_required
    early-return with the contract-row / operator-override carve-out.
  • controller/src/config/mod.rs + controller/src/main.rs — new tests.

Curl evidence (after fix)

$ CONTROLLER_WORKLOADS=deploy/configs/mvp-workload.yaml \
  USE_TYPED_STAGE_SPLIT=1 ./controller &
$ curl -s http://localhost:8080/api/v1/collector-config/agent \
       -H 'X-Agent-Runtime: AsapOtel' | grep metric.name
    - statement: route() where metric.name == "endpoint_request_freq"
    - statement: route() where metric.name == "http_requests_total_latency_ms"
    - statement: route() where metric.name == "request_size_bytes"
    - statement: route() where metric.name == "top_endpoint_qps"
    - statement: route() where metric.name == "unique_users_per_min"
    - statement: route() where metric.name == "http_freshness_probe_warm"
    - statement: route() where metric.name == "http_freshness_probe_archive"

5 sketched metric routes (DDSketch / KLL / HLL / CountSketch /
CountMinSketch) + 2 freshness probes — full 5-sketch coverage.

Test plan

  • cargo test --release --bin controller — 608 passed (was 603 on
    origin/main; +5 new tests, +0 regressions; same 10 pre-existing failures).
  • New regression bootstrap_routing_table_covers_all_five_sketches_for_live_mvp_yaml
    pins the contract: registry-walk path produces a 5-entry routing
    table for the live deploy/configs/mvp-workload.yaml shape.
  • Live curl against a controller pointed at the deployed YAML
    confirms all 5 sketched metrics + both freshness probes route correctly.
  • live_mvp_workload_yaml_loads_with_overrides smoke-tests the
    actual deploy file deserialises with all four overrides intact.

Closes #46.

🤖 Generated with Claude Code

…Sketch/CountMin) (#46)

Two stitches were missing for MVP §46 entries 5–8 — the typed planner had
to be reached with both the operator's `sketch_family_override` *and* the
parsed-query `exact_required` flag turned off:

1. `WorkloadEntry` (config/workloads.rs) silently ignored the
   `sketch_family_override` / `target_path` keys (serde unknown-field
   default) so the registry-walk pre-pop in `main()` always set
   `QuerySpec.sketch_type = None`. Add the field with a case-insensitive
   `SketchType` deserialiser (the live YAML spells `KLL` / `HLL` /
   `CountSketch` / `CountMinSketch` mixed-case while the
   `#[serde(rename_all = "lowercase")]` enum would otherwise reject
   them) and thread it through `main`'s pre-pop loop.

2. `bind_workload_typed` (planner/rules.rs:81) early-returned `None`
   on `w.exact_required = true`. The query parser sets that flag when
   the inner walk synthesises a bare-VectorSelector → `AggFunc::Sum`
   (the path `count(metric)`, `topk(K, metric)`, and
   `rate(metric[5m])` all hit) — so for entries 6–8 the typed binder
   declined before the metric-name match ever ran. Carve out an
   exception: when the metric is a contract row
   (`classify_demo_metric` returns `Some`) or carries an explicit
   `sketch_type_override`, those signals win over the parser's
   `exact_required` flag because both are explicit "sketch this
   metric" instructions from the operator/contract.

Tests added:

- `config::workloads::tests::deserialize_sketch_family_override_*` —
  deserialisation accepts both mixed-case (`KLL`) and lowercase /
  alias spellings (`countmin`, `cms`).
- `config::workloads::tests::live_mvp_workload_yaml_loads_with_overrides`
  — smoke-test the live `deploy/configs/mvp-workload.yaml`.
- `config::runtime_tests::collect_metric_to_family_binds_all_six_…`
  — registry-walk path produces a 5-entry routing table (5 sketches,
  raw declines).
- `api_tests::bootstrap_routing_table_covers_all_five_sketches_for_live_mvp_yaml`
  — end-to-end via the bootstrap GET endpoint, with metric names
  exactly as they appear in the live demo YAML.

Curl evidence (controller running with `CONTROLLER_WORKLOADS=
deploy/configs/mvp-workload.yaml USE_TYPED_STAGE_SPLIT=1`):

  routing:
    - route() where metric.name == "endpoint_request_freq"            → metrics/countminsketch_path
    - route() where metric.name == "http_requests_total_latency_ms"   → metrics/ddsketch_path
    - route() where metric.name == "request_size_bytes"               → metrics/kll_path
    - route() where metric.name == "top_endpoint_qps"                 → metrics/countsketch_path
    - route() where metric.name == "unique_users_per_min"             → metrics/hll_path
    - route() where metric.name == "http_freshness_probe_warm"        → metrics/raw_passthrough
    - route() where metric.name == "http_freshness_probe_archive"     → metrics/raw_passthrough

before this change the routing table only carried the DDSketch and KLL
rows; HLL / CountSketch / CountMinSketch silently dropped.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@zzylol
zzylol merged commit 9f638b9 into main May 8, 2026
zzylol added a commit that referenced this pull request May 8, 2026
…equests_total_latency_ms) (#346)

PR #337's SKETCH_FAMILIES table used `http_latency_ms` as the DDSketch
metric, but the actual workload-spec entry (and fake-exporter emit) is
`http_requests_total_latency_ms`. Result: post-PR-#345 demo runs reported
DDSketch row n=0 even though warm tier was answering quantile queries —
the rel-err values were attributed to no family.

This aligns the table with the live workload-spec.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@zzylol
zzylol deleted the fix/stitching-binds-hll-cs-cms branch May 9, 2026 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MVP demo: test-first validation of ASAPCollector + ASAPQuery-backend

1 participant