refactor(deploy): retire gateway from default data path — agent → asapquery-backend - #377
Merged
zzylol merged 1 commit intoMay 15, 2026
Conversation
…pquery-backend The default data path is now: fake-exporter → agent → asapquery-backend collapsing the former: fake-exporter → agent → gateway → asapquery-backend Why this works: asapquery-backend's `--enable-otel-ingest` already accepts sketch data points and merges them per-aggregation_id via its precompute engine accumulators (DDSketchAccumulator / HLLAccumulator / etc.) — no middle-tier OTel gateway merge processor is required. The asapquery-backend control-plane PRs #241 / #242 / #243 retire the gateway-targeted emit pieces (legacy backend-collector emitter + shadow `BackendCollectorConfig` + agent OTLP default_host). Code kept-in-source per "keep gateway impl in repos" intent: - All `configs/gateway*.yaml` files preserved - `configs/asap/asap-otel-gateway-mvp-placeholder.yaml` preserved - Gateway service definition in `base.yml` profile-gated under `["gateway-legacy"]` — `docker compose --profile gateway-legacy …` still spins it up - `gateway_up` / `gateway_down` functions in multinode `run_demo.sh` defined-but-uncalled (lifecycle dropped from `arm_up` / `arm_down`) Files touched: Agent OTel configs (rename `otlp/gateway` exporter block to `otlp/backend`, change `endpoint: gateway:4317` → `endpoint: backend:4317`): - mvp-singlenode/configs/asap-otel-agent-{b0a-raw-stream, b0b-raw-batched, b2-full, b3-delta, b4-tunable, b6-asap-single-sketch}.yaml - mvp-multinode/configs/asap/asap-otel-agent-b6-asap-single-sketch.yaml Workload tags (assign_to_role: backend|gateway → agent): - mvp-singlenode/configs/workloads.yaml (2 rows) - mvp-singlenode/configs/mvp-workload.yaml (1 row) - mvp-multinode/configs/asap/mvp-workload.yaml (1 row) Compose: - mvp-singlenode/docker-compose/base.yml — gateway service `profiles: ["gateway-legacy"]`; fake-exporter default EXPORTER_TARGET=backend:4317; fake-exporter depends_on→backend - mvp-singlenode/docker-compose/e2e-overlay.yml — gateway override also `profiles: ["gateway-legacy"]` - mvp-singlenode/docker-compose/agents-N{1,10,100}.yml — agent depends_on→backend (was gateway) - mvp-singlenode/docker-compose/gen-agents.sh — template emits same - mvp-singlenode/docker-compose/mvp-multi-stage.yml — gateway override `profiles: ["gateway-legacy"]`; comment notes how to re-enable multi-stage flow (requires `--profile gateway-legacy` + gateway- targeting AGENT_CONFIG) Multinode orchestration: - mvp-multinode/topology.env — NODE1 reclassified "unused / reserved" (DNS alias preserved for opt-in) - mvp-multinode/scripts/run_demo.sh — gateway_up/down dropped from arm lifecycle (functions remain) - mvp-multinode/scripts/run_demo_sweep.sh — roles map node1 "gateway" → "unused" Docs: - mvp-singlenode/README.md — port table + topology blurb updated - mvp-multinode/README.md — topology diagram updated Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Collapses the default data path from
fake-exporter → agent → gateway → asapquery-backendtofake-exporter → agent → asapquery-backend. asapquery-backend's--enable-otel-ingestalready accepts sketch data points and merges them per-aggregation_id via its precompute engine accumulators, so no middle-tier OTel gateway merge processor is needed.Aligns with control-plane PRs:
BackendCollectorConfigbackend:4317Kept-in-source per ”keep gateway impl in repos” intent
configs/gateway*.yamlfiles preservedconfigs/asap/asap-otel-gateway-mvp-placeholder.yamlpreservedbase.ymlprofile-gated under[”gateway-legacy”]— still accessible viadocker compose --profile gateway-legacy …gateway_up/gateway_downfunctions in multinoderun_demo.shdefined-but-uncalledWhat changed (22 files)
Agent OTel configs (7 files):
otlp/gatewayexporter block →otlp/backend,endpoint: gateway:4317→backend:4317.Workload tags (4 rows across 3 files):
assign_to_role: backend|gateway→agent.Compose (7 files):
base.yml— gateway serviceprofiles: [gateway-legacy];fake-exporterdefault target → backend;depends_on→ backend.e2e-overlay.yml— gateway override also profile-gated.agents-N{1,10,100}.yml+gen-agents.shtemplate — agentdepends_on→ backend.mvp-multi-stage.yml— gateway override profile-gated; comment explains how to re-enable multi-stage flow.Multinode orchestration (3 files):
topology.env— NODE1 reclassified ”unused / reserved” (DNS alias kept for opt-in).run_demo.sh—gateway_up/downdropped fromarm_up/arm_down(functions kept).run_demo_sweep.sh— roles-map node1: ”gateway” → ”unused”.Docs (2 files): topology diagram + port table.
Test plan
docker compose -f base.yml -f agents-N1.yml up— confirm agent comes up, exports tobackend:4317, asapquery-backend receives + accumulates sketches.docker compose -f base.yml -f e2e-overlay.yml -f agents-N1.yml up— confirm e2e flow without gateway.docker compose --profile gateway-legacy -f base.yml -f agents-N1.yml up— confirm gateway service still spins up under opt-in profile.run_demo.sh asap— confirm arm runs end-to-end with node1 idle.🤖 Generated with Claude Code