Skip to content

control-plane: emit fused asap_edge edge config - #311

Merged
zzylol merged 1 commit into
mainfrom
feat/controller-emit-asap-edge
May 23, 2026
Merged

zzylol merged 1 commit into
mainfrom
feat/controller-emit-asap-edge

Conversation

@zzylol

@zzylol zzylol commented May 23, 2026

Copy link
Copy Markdown
Contributor

Summary

The edge agent replaced the old routing-connector per-family fan-out (DDSketch/KLL/HLL/CountSketch/CountMinSketch + raw passthrough + gorillas3 cold processor) with ONE fused asap_edge processor in a single pipeline. This teaches the control-plane to emit that shape.

New emit_edge_yaml_asap_edge(cfg, opamp_endpoint, agent_id) produces:

otlp -> [memory_limiter, cumulativetodelta, asap_edge] -> otlp/backend

with a single metrics pipeline, opamp extension, and an asap_edge processor carrying shard_count, window_duration, drop_original, a metrics[] map, and a cold: block — matching the hand-written contract in deploy/mvp-multinode/configs/asap/asap-otel-agent-b6-asap-single-sketch.yaml.

Mapping (all from existing EdgeStageConfig fields)

  • sketch families: metric_to_family x sketch_processors params -> per-entry relative_accuracy (ddsketch) / k (kll) / nothing (hll) / rows+cols (countsketch, countminsketch), mirroring build_edge_processor_block.
  • sum family: every AggRole::Sum metric (in cumulative_counter_metrics) that has grouping labels declared and is NOT routed to a sketch family -> { metric, family: sum, aggregate_by: [<group_by_labels>] }. Same predicate the routing path's metrics/sum_aggregate selection uses; the group-by labels come from metric_to_grouping_labels (sourced from workload.group_by_labels).
  • cumulativetodelta.include.metrics: cumulative_counter_metrics (the sum metrics plus the counter-shaped sketch inputs), match_type: strict.
  • cold block: enabled when the plan declared any archive_tier_metrics; block_duration from the archive window.

Wiring

emit_edge_yaml dispatches to the fused emitter when ASAP_EDGE_FUSED is set (default off), else keeps the 5-sketch routing emit. Env-gate (mirroring typed_stage_split_enabled() / ASAP_AGENT_MEMORY_LIMIT_MIB) is used instead of a new EdgeStageConfig field so the change is additive: no struct-literal churn across ~17 construction sites, no serde wire-shape bump, both paths read identical cfg fields, and the entire existing routing-emit test surface is unchanged. The flag is the canonical migration switch; the default flips to on once the fused agent build is the default deployment (and the routing path is retired).

Reported gap (not fabricated)

EdgeStageConfig does NOT carry a cold ship_endpoint or an external_labels map today (only the S3/MinIO knobs build_gorillas3_yaml reads from controller env). Following the emitter's documented-placeholder convention for unresolved deploy targets (resolve_export_endpoint), ship_endpoint is derived from the OTLP backend host (http://<host>:9098/ingest/gorilla) and external_labels.cluster reads ASAP_CLUSTER (default asap-mvp). Threading a real per-deploy cold endpoint + external-label map needs a new EdgeStageConfig field populated from the plan -- left as a follow-up (clearly commented in code).

Test plan

  • cargo build -p control_plane clean (no new warnings)
  • cargo test -p control_plane --lib emit::stage_config (84 pass, incl. 2 new)
  • cargo test -p control_plane --lib (769 pass)
  • New fused_asap_edge_emits_single_pipeline_and_metrics_list: YAML parses, single metrics pipeline [memory_limiter, cumulativetodelta, asap_edge], sum-by + 5 sketch metrics[] entries with correct params, cold block present/enabled, cumulativetodelta strict include list
  • New fused_gate_off_keeps_routing_shape: gate-off keeps the routing-connector shape (backward compat)

Generated with Claude Code

The agent replaced the routing-connector per-family fan-out with one
fused `asap_edge` processor in a single pipeline. Emit that shape:
`otlp -> [memory_limiter, cumulativetodelta, asap_edge] -> otlp/backend`,
with the per-metric family map (sum/ddsketch/kll/hll/countsketch/
countminsketch), the Sum-by aggregate_by entries, the cumulativetodelta
counter list, and a cold archive block.

Gated on `ASAP_EDGE_FUSED` (default off) so the established 5-sketch
routing emit and its test surface are unchanged until the fused agent
build is the default deployment; this keeps the change additive (no
EdgeStageConfig struct-literal churn) and reads the identical cfg fields.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@zzylol
zzylol merged commit 42a3ba2 into main May 23, 2026
@zzylol
zzylol deleted the feat/controller-emit-asap-edge branch July 17, 2026 20:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant