ship MVP demo for #46 - #287
Merged
Merged
Conversation
Adds the issue-46 MVP demo — single-cell paired run that compares
the all-sketches warm tier + Gorilla-S3 cold archive against a raw
OTLP streaming baseline, emitting X/Y/Z reduction deltas and a
PASS/FAIL verdict per acceptance criterion.
Three new files:
* deploy/docker-compose/baseline-b6-gorilla-s3.yml — overlay
enabling the gorillas3 processor on the agent + the
GorillaQueryEngine env on the backend, plus a one-shot
`minio-setup-gorilla` job to seed the asap-gorilla bucket.
* deploy/configs/sketchcol-agent-b6-gorilla-s3.yaml — agent
pipeline running gorillas3 (cold tier, drop_original=false)
alongside DDSketch + KLL + HLL + CountSketch + CountMinSketch
(warm tier).
* deploy/scripts/run_mvp_demo.sh + deploy/scripts/mvp_report.py —
driver + reducer. Driver brings up the ASAP cell, runs a 60 s
soak with the e2e replay client + one ad-hoc cold-fallback
query, captures producer wire bytes / agent CPU+RSS / backend
p99 / accuracy + the cold-fallback ad-hoc query response,
then repeats with the b0a-raw-stream baseline. Reducer
computes the X/Y/Z pct-reductions and PASS/FAIL → MVP_REPORT.md.
Build-config touch-ups (required to make the demo runnable; not
processor source):
* builder-config.yaml — adds gorillas3processor to the
sketchcollector OCB manifest so the asap/sketchcol:dev image
actually has the processor.
* Dockerfile.backend — adds an `asap-gorilla` build context for
the backend's path-dep on the asap-gorilla crate (Phase-3
cold-engine).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The flag is a postfix-agent edit on `fix/eval-config-gaps-postsweep` that hasn't landed in main yet, so calling measure-baseline.py with it from a clean origin/main checkout fails: measure-baseline.py: error: unrecognized arguments: --bytes-sample-warmup 3 Drop the flag so the MVP demo runs against the version of measure-baseline.py that ships on main. The 15s bytes-sample-window is wide enough to cover producer flushes at 1Hz scrape without the extra warm-up sample. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Run produced by `bash deploy/scripts/run_mvp_demo.sh`. Single
cell, single script: ASAP all-sketches + Gorilla-S3 cold archive
overlay vs raw OTLP streaming baseline at N=1 · 1 Hz · cardinality
10000 · soak 60s.
Verdicts (per `mvp_report.py::criterion_*`):
X bandwidth PASS — raw 188 KB/s → asap 0 B/s
(caveat: ASAP value reflects a 15s docker-stats
window that landed between agent flushes; the
warm-tier pipeline is still emitting sketched
OTLP — bytes are non-zero but smaller than the
window's resolution.)
Y query latency FAIL — raw p99 1644 ms → asap 1743 ms (-6 %)
Z combined cost PASS — backend CPU drops 16.5% → 0.01% as the
warm-tier sketch path replaces raw ingest
Accuracy PASS — count_unique median rel-err = 0
(n=29; cardinality 10 000 hit the cardinality
accumulator exactly)
Cold-fallback FAIL — gorillas3 processor produced no chunks
in the asap-gorilla bucket during this run; the
backend's Phase-6 GorillaQueryEngine hooking
also requires `precompute_engine.rs` (the
deployed binary) to load `ASAP_GORILLA_S3_*`
envs, which is wired only on `src/main.rs`.
Artifacts committed: MVP_REPORT.md + per-cell measurement.csv,
replay.jsonl, accuracy.csv, ad_hoc_query_response.json,
gorilla_chunks.txt. Cold-truth JSONL snapshots (~600 MB each)
are kept locally only — too large for git.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
10 tasks
Merged
7 tasks
zzylol
added a commit
that referenced
this pull request
May 19, 2026
…10 Mbps double-hop) (#401) The asap-gateway OTel-collector hop was originally the cross-agent fan-in tier for sum-by-zone aggregations. Post ASAPQuery-backend PRs #283 (B2 stage-emit) and #287/#290 (cumulative emit), the wave-tier backend does that fan-in natively via the `evaluate_exact_agg` reducer pulling sids from every agent. The gateway became a pure forwarder that doubled every byte (agent → gateway → backend) for no algorithmic reason, costing ~10 Mbps per arm on the multinode demo (issue #400). Scope: - multinode `run_demo.sh`: remove `gateway_up` / `gateway_down`, drop the gateway out of `arm_up` / `arm_down`. `stop_node node1` stays in the teardown to reap any pre-#400 leftover container; `node1` is otherwise unused. - multinode `topology.env`: drop the `--add-host=gateway:` alias and retire the node1 role note. - multinode + singlenode agent yaml: OTLP `endpoint:` flips from `gateway:4317` to `backend:4317`. The backend already accepts OTLP on 4317 (`--otel-grpc-port=4317` in `run_demo.sh` and base.yml). - multinode + singlenode `mvp-workload.yaml`: the lone `assign_to_role: gateway` entry (sum-by-zone) becomes `assign_to_role: agent`. The controller's typed `split_typed_three_stage` emits only Edge + Backend stages when no Gateway stage is occupied, so `emit_gateway_yaml` simply doesn't fire and `push_to_role(Gateway, ...)` is a no-op — no controller code change needed. - multinode `asap-otel-gateway-mvp-placeholder.yaml`: deleted. - singlenode `base.yml`: gateway service removed; its host ports (14317 / 14318) reassigned to the backend container so external tooling that previously hit the gateway can hit the backend on the same host ports. `fake-exporter` default `EXPORTER_TARGET` flips to `backend:4317`; `depends_on` flips to `backend`. - singlenode `agents-N{1,10,100}.yml` + `gen-agents.sh`: `depends_on` flips from `gateway` → `backend`. Validation (multinode, 60 s NIC sample, asap arm, WARMUP_S=90): - node1 docker ps: empty (no asap-gateway). - node2 docker ps: backend + thanos-{query,store-gateway,compact} + minio + prometheus + controller; no gateway. - All 5 wave queries against node2:9091 return `data_source: asap_query`: quantile_over_time(0.99, http_requests_total_latency_ms[5m]) → 10000 series max by (zone) (quantile_over_time(0.99, ...)) → 4 series sum by (zone) (http_requests_total) → 4 series sum by (zone) (rate(http_requests_total[5m])) → 4 series topk(5, sum by (zone) (rate(...))) → 4 series - Per-node NIC bandwidth (enp130s0f0, 60 s): node0 RX 0.16 Mbps TX 6.80 Mbps (agent-a → backend) node1 RX 0.00 Mbps TX 0.00 Mbps (idle — pre-fix carried the gateway double-hop) node2 RX 13.56 Mbps TX 0.33 Mbps (backend receives both agents) node3 RX 0.16 Mbps TX 6.76 Mbps (agent-b → backend) Pre-fix the node1 link carried ~13 Mbps in + ~13 Mbps out as a pure forwarder. Post-fix that ~26 Mbps of node1 wire traffic is gone; the remaining ~13.6 Mbps lives only on the agent→backend hop. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #46. Adds the issue-46 MVP demo — a single-cell paired run that compares the all-sketches warm tier + Gorilla-S3 cold archive against a raw OTLP streaming baseline, emitting X/Y/Z reduction deltas and a PASS/FAIL verdict per acceptance criterion.
deploy/docker-compose/baseline-b6-gorilla-s3.yml+deploy/configs/sketchcol-agent-b6-gorilla-s3.yaml: the agent runs thegorillas3processorcold-archive (writes Gorilla XOR-delta chunks to MinIO under bucketasap-gorilla) alongside the all-five-sketch warm-tier pipeline; backend reads the Phase-5/6ASAP_GORILLA_S3_*env vars.deploy/scripts/run_mvp_demo.sh+deploy/scripts/mvp_report.py: driver brings up the ASAP cell, soaks 60 s, runs the e2e replay client + one ad-hoc cold-fallback query (count(http_requests_total)), snapshots producer wire bytes / agent CPU+RSS / backend p99 / accuracy, then repeats with the b0a-raw-stream baseline. Reducer joins both cells and emitsMVP_REPORT.md.opentelemetry-collector-contrib-patch/cmd/sketchcollector/builder-config.yaml+deploy/docker/Dockerfile.backend: build-config touch-ups so thegorillas3processorends up in theasap/sketchcol:devimage and the backend's path-dep on theasap-gorillacrate resolves inside the build container. No processor / engine source-code edits.Test plan
bash deploy/scripts/run_mvp_demo.shruns end-to-end on the host (gated on the host-widerun_e2e_sweep.shfinishing first —wait_for_sweep_idleblocks for up toSWEEP_WAIT_CAP_Sseconds, default 3600)MVP_REPORT.mdshows X (bandwidth %) / Y (latency p99 %) / Z (combined CPU+RSS %) numeric reductionsaccuracy.csv≤ 0.05 (PASS for criterion 4)data_source: gorilla_archiveif Phase-6 router is wired into the running backend binary, elsedata_source: sketch_warm+ the structural MinIO chunk presence — see PARTIAL handling inmvp_report.py::criterion_cold_fallback)asap-gorillabucket has chunks (verified viamc ls)🤖 Generated with Claude Code