feat: capability-miss → ControllerClient call-out (PR G)

[zzylol]

PR G — closes the query plane's side of the feedback loop with the DataCollector controller.

The problem this solves

Today when SimpleEngine gets a query it can't answer from precomputed state — streaming_config.find_compatible_aggregation(&requirements) returns None — it silently falls through to the §5.2 fallback (direct Prometheus read, SQL forwarding) and returns whatever that gets. The controller never learns that the workload wants a sketch it doesn't currently have, so the gap persists across every future query of the same shape.

What this PR adds

Three call sites in simple_engine.rs (two SQL, one PromQL) now go through a helper:

fn find_compatible_aggregation_with_miss_notify(&self, requirements: &QueryRequirements)
    -> Option<AggregationIdInfo>
{
    let result = self.streaming_config.find_compatible_aggregation(requirements);
    if result.is_none() {
        spawn_capability_miss_notify(&self.controller_client, requirements);
    }
    result
}

On miss, the helper fires a fire-and-forget tokio::spawn task that POSTs a JSON payload (metric, statistics, data range, grouping labels, spatial filter) to the DataCollector controller's plan endpoint. The current query continues straight into the §5.2 fallback — it is not retried, not delayed, not blocked on the controller response.

The new pieces:

• drivers/query/controller_client.rs (new module)

  • trait ControllerClient { async fn notify_capability_miss(&QueryRequirements) -> Result<(), String>; }
  • HttpControllerClient — HTTP impl with a 5s timeout so a slow or unreachable controller can't stall the spawned task. POSTs a flat JSON DTO projection of QueryRequirements — using a flat DTO avoids adding Serialize derives to shared asap_types / promql_utilities crates.
  • spawn_capability_miss_notify(&Option<Arc<dyn ControllerClient>>, &QueryRequirements) — fire-and-forget helper. Spawns via tokio::spawn. No-op when the option is None. Errors logged at WARN — capability misses must never fail the query.
  • MockControllerClient exported from tests so follow-up integration tests can compose a SimpleEngine with a fake client without needing an HTTP mock.

• SimpleEngine::with_controller_client(...) builder — new field controller_client: Option<Arc<dyn ControllerClient>>, unset by default, so existing code paths are behaviorally unchanged.

• --controller-endpoint <URL> CLI flag on main.rs. Default: unset. When set, main.rs constructs an HttpControllerClient and attaches it via SimpleEngine::with_controller_client(...).

How this closes the e2e loop with PR #10

PR #10 gave the backend a POST /api/v1/streaming-config endpoint so an external process can push a new StreamingConfig at runtime. This PR gives the controller a reason to call it:

query arrives → SimpleEngine miss
    ↓
PR #11 fires fire-and-forget POST to controller
    ↓
current query returns §5.2 fallback answer
    ↓
controller's replanner generates a new sketch plan
    ↓
controller POSTs the new plan to backend via PR #10's endpoint
    ↓
next query for the same shape finds a precomputed match

Why fire-and-forget instead of retry

Retrying the query after the plan lands would require waiting on:

1. Controller plan-generation latency
2. The push back to the backend
3. At least one flusher tick before the new aggregation shows up in the worker pool

That coordination is out of scope for PR G. The fire-and-forget approach treats misses as telemetry that closes the loop over multiple query events rather than within a single query — the §5.2 fallback remains the correctness anchor, and the loop pays off on the second identical query, not the first.

Tests

Five new unit tests in controller_client.rs:

1. payload_projects_all_requirement_fields — DTO conversion from QueryRequirements round-trips through JSON correctly
2. spawn_helper_is_noop_when_client_is_none — safe to call with None
3. spawn_helper_invokes_client_via_tokio_spawn — mock client confirms the fire-and-forget path fires end-to-end
4. http_client_reports_non_success_status — real axum mock returning 500 → client maps to formatted Err
5. http_client_success_path_round_trips_payload — real axum mock parses the posted JSON and asserts field-by-field

Drive-by

Same two clippy-on-rust-1.91 issues in persistence code that PR #9 and PR #10 also fix:

• persistence/cache.rs — unnecessary u64 as u64 cast
• persistence/part.rs — test &[snap.clone()] → std::slice::from_ref(&snap)

Harmless duplicate when PR #9 lands first.

Validation

• ✅ cargo check --all-targets: clean
• ✅ cargo clippy --all-targets -- -D warnings: clean
• ✅ cargo fmt --check: clean
• ✅ cargo test -p query_engine_rust --lib: 479 passed (up 5 from main — the new unit tests)

Stack

• PR #9 — PR I: MessagePack encoding parity
• PR #10 — PR E: StreamingConfig hot-reload phase 1 (the controller's push target for the plan generated in response to this PR's notification)
• PR feat: capability-miss → ControllerClient call-out (PR G) #11 (this PR) — PR G: capability-miss → ControllerClient notification
• DataCollector #154 — proto MSGPACK enum names
• DataCollector #155 — PR F: OpAMP config push architecture docs

Follow-ups

• Query retry — wire an optional retry after the plan lands and the backend's StreamingConfig reflects the new plan. Needs PR E phase 2 (per-query re-snapshot in SimpleEngine). Retry-once, bounded wait, fall back to §5.2 on timeout.
• Deduplication / rate-limiting — LRU + "seen in last N seconds" filter to prevent the same requirements from spawning notifications repeatedly under load.
• Retry telemetry — (miss → notify → plan landed) latency tracking to tell whether the loop closes fast enough to matter.

🤖 Generated with Claude Code