control-plane: emit fused asap_edge edge config - #311
Merged
Merged
Conversation
The agent replaced the routing-connector per-family fan-out with one fused `asap_edge` processor in a single pipeline. Emit that shape: `otlp -> [memory_limiter, cumulativetodelta, asap_edge] -> otlp/backend`, with the per-metric family map (sum/ddsketch/kll/hll/countsketch/ countminsketch), the Sum-by aggregate_by entries, the cumulativetodelta counter list, and a cold archive block. Gated on `ASAP_EDGE_FUSED` (default off) so the established 5-sketch routing emit and its test surface are unchanged until the fused agent build is the default deployment; this keeps the change additive (no EdgeStageConfig struct-literal churn) and reads the identical cfg fields. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The edge agent replaced the old
routing-connector per-family fan-out (DDSketch/KLL/HLL/CountSketch/CountMinSketch + raw passthrough +gorillas3cold processor) with ONE fusedasap_edgeprocessor in a single pipeline. This teaches the control-plane to emit that shape.New
emit_edge_yaml_asap_edge(cfg, opamp_endpoint, agent_id)produces:with a single
metricspipeline,opampextension, and anasap_edgeprocessor carryingshard_count,window_duration,drop_original, ametrics[]map, and acold:block — matching the hand-written contract indeploy/mvp-multinode/configs/asap/asap-otel-agent-b6-asap-single-sketch.yaml.Mapping (all from existing
EdgeStageConfigfields)metric_to_familyxsketch_processorsparams -> per-entryrelative_accuracy(ddsketch) /k(kll) / nothing (hll) /rows+cols(countsketch, countminsketch), mirroringbuild_edge_processor_block.AggRole::Summetric (incumulative_counter_metrics) that has grouping labels declared and is NOT routed to a sketch family ->{ metric, family: sum, aggregate_by: [<group_by_labels>] }. Same predicate the routing path'smetrics/sum_aggregateselection uses; the group-by labels come frommetric_to_grouping_labels(sourced fromworkload.group_by_labels).cumulative_counter_metrics(the sum metrics plus the counter-shaped sketch inputs),match_type: strict.archive_tier_metrics;block_durationfrom the archive window.Wiring
emit_edge_yamldispatches to the fused emitter whenASAP_EDGE_FUSEDis set (default off), else keeps the 5-sketch routing emit. Env-gate (mirroringtyped_stage_split_enabled()/ASAP_AGENT_MEMORY_LIMIT_MIB) is used instead of a newEdgeStageConfigfield so the change is additive: no struct-literal churn across ~17 construction sites, no serde wire-shape bump, both paths read identical cfg fields, and the entire existing routing-emit test surface is unchanged. The flag is the canonical migration switch; the default flips to on once the fused agent build is the default deployment (and the routing path is retired).Reported gap (not fabricated)
EdgeStageConfigdoes NOT carry a coldship_endpointor anexternal_labelsmap today (only the S3/MinIO knobsbuild_gorillas3_yamlreads from controller env). Following the emitter's documented-placeholder convention for unresolved deploy targets (resolve_export_endpoint),ship_endpointis derived from the OTLP backend host (http://<host>:9098/ingest/gorilla) andexternal_labels.clusterreadsASAP_CLUSTER(defaultasap-mvp). Threading a real per-deploy cold endpoint + external-label map needs a newEdgeStageConfigfield populated from the plan -- left as a follow-up (clearly commented in code).Test plan
cargo build -p control_planeclean (no new warnings)cargo test -p control_plane --lib emit::stage_config(84 pass, incl. 2 new)cargo test -p control_plane --lib(769 pass)fused_asap_edge_emits_single_pipeline_and_metrics_list: YAML parses, singlemetricspipeline[memory_limiter, cumulativetodelta, asap_edge], sum-by + 5 sketchmetrics[]entries with correct params, cold block present/enabled, cumulativetodelta strict include listfused_gate_off_keeps_routing_shape: gate-off keeps the routing-connector shape (backward compat)Generated with Claude Code