control-plane: thread cold ship_endpoint + external_labels into EdgeStageConfig - #313
Merged
Merged
Conversation
…tageConfig PR #311's emit_edge_yaml_asap_edge derived the fused asap_edge cold tier's ship_endpoint from the OTLP backend host (http://<backend>:9098/ingest/gorilla) and read external_labels.cluster from ASAP_CLUSTER inline, because EdgeStageConfig carried neither field. Both were wrong: the cold tier ships to the gorilla-merger over HTTP ingest port 10908 (gRPC 10907), not backend:9098. Add cold_ship_endpoint: Option<String> and cold_external_labels: Vec<(String,String)> to EdgeStageConfig and read them in the emitter instead of the inline placeholder. The colored_dag L5 layer is deployment-independent (no DeploymentConstraints plumbed in), so it cannot resolve a real per-deploy endpoint; it populates the single named defaults (default_cold_ship_endpoint / default_cold_external_labels), which a deploy-aware layer can override post-emit the same way exporter_target is. All struct-literal construction sites updated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #311. The merged
emit_edge_yaml_asap_edgederived the fusedasap_edgecold tier target from a placeholder:ship_endpointcame from the OTLP backend host (http://<backend>:9098/ingest/gorilla) andexternal_labels.clusterwas read inline fromASAP_CLUSTER— becauseEdgeStageConfigcarried neither field. Both were wrong: the cold tier ships to the gorilla-merger over HTTP ingest port 10908 (gRPC 10907), notbackend:9098.cold_ship_endpoint: Option<String>andcold_external_labels: Vec<(String, String)>toEdgeStageConfig(physical/colored_dag/emitter.rs).default_cold_ship_endpoint()(http://gorilla-merger:10908/ingest/gorilla) anddefault_cold_external_labels()(cluster=<ASAP_CLUSTER|asap-mvp>) — and route both the emit fallback and the L5 build site through them (no more inline guess).emit_edge_yaml_asap_edgenow reads the threaded fields (falling back to the named defaults when unset).ASAP_EDGE_FUSEDgating untouched.EdgeStageConfigstruct-literal site (~18 across 6 files) so the crate compiles. Thecolored_dagL5 build site uses the named defaults because that layer is deployment-independent (noDeploymentConstraintsplumbed in — reported, not fabricated); a deploy-aware layer can overwriteedge.cold_*post-emit the same wayexporter_targetis.Threading-from-plan note
The
colored_dagL5 emitter is explicitly deployment-independent — it has no access to a real per-deploy cold endpoint or cluster name. So the L5 site populates the named defaults (gorilla-merger:10908,cluster=asap-mvp), not a plan-resolved value. The new fields make it trivial for a future deploy-aware caller to thread real values; this PR fixes the wrong host/port and removes the inline placeholder.Test plan
cargo build -p control_planeclean — no new warnings vsorigin/mainbaseline (lib 4 / bin 2 warnings, identical set, all pre-existing).cargo test -p control_plane --lib emit::stage_config— 84 pass.cargo test -p control_plane --lib physical::colored_dag— 21 pass.fused_asap_edge_emits_single_pipeline_and_metrics_listto threadcold_ship_endpoint = http://gorilla-merger:10908/ingest/gorilla+cluster=asap-mvpand assert the emitted cold block surfaces exactly that (and never emits:9098).--libsuite passes single-threaded (769/769). NOTE: the full parallel--librun flakes on a pre-existingASAP_EDGE_FUSEDprocess-globalstd::envrace introduced by control-plane: emit fused asap_edge edge config #311 (its fused emit test sets the env var while parallel mvp46 tests read it without the shared lock) — reproduced identically on the untouchedorigin/mainbaseline (5/8 parallel failures there). Out of scope for this PR; not caused by this change.