benchmark: q9 ground truth - #128
Closed
GnaneshGnani wants to merge 91 commits into
Closed
GnaneshGnani wants to merge 91 commits into
GnaneshGnani wants to merge 91 commits into
Conversation
- ground_truth/q4.py: add numpy import - compare.py: frac_hilo_lt_2pct (p0 vs low, p100 vs high) - run.py: default batch-size 50 - QUERY_PORTING_GUIDE.md: Q4 documentation
The guide stays on disk as an untracked local file.
zzylol
pushed a commit
that referenced
this pull request
May 11, 2026
…for HLL Two Phase-1-era staleness bugs in the replay/e2e query JSONs: 1. Per-sketch metric-name suffixes (_quantile, _kll, _hll) were dropped in #372 — the backend ingests under raw input names and the OTLP pdata variant tag carries the sketch type. The query JSONs still referenced the legacy suffixed names `http_requests_total_latency_ms_{quantile,kll,hll}`. Switched to the raw metric names per `mvp-workload.yaml`: - DDSketch quantile → http_requests_total_latency_ms - KLL quantile → http_requests_total_latency_ms - HLL cardinality → unique_users_per_min 2. The HLL JSON used `count_over_time(metric[r])` which counts SAMPLES, not distinct cardinality. Replaced with MetricsQL's `distinct_over_time(metric[r])` — the natural primitive for the HLL cardinality estimate. See https://docs.victoriametrics.com/victoriametrics/metricsql/#distinct_over_time CMS / CS JSONs already use the raw name `http_requests_total`; left unchanged. NOTE: the backend's warm-tier analyzer (warm_tier_analysis.rs in controller, PR #128) does not yet recognize `distinct_over_time` — it accepts `count_distinct_over_time` / `cardinality_estimate` / `count_distinct`. The analyzer-side alias landing is a follow-up that pairs naturally with the Prometheus → VictoriaMetrics swap; until that lands, the HLL queries here will surface as `UnsupportedFunction("distinct_over_time")` → archive forward, which is the safe fallback. Both `deploy/scripts/` (canonical) and `deploy/mvp-multinode/scripts/` (rsync source) updated to keep them in sync — the multinode orchestrator rsyncs deploy/scripts/ to each node at Phase 0 anyway, but the multinode copy is committed to make the 4-node demo reproducible from a fresh clone. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zzylol
added a commit
that referenced
this pull request
May 11, 2026
* feat(deploy): 4-node MVP demo orchestrator under version control The 4-node MVP demo driver was built in earlier sessions as local-only scratch under /mydata/mvp-multinode/. Bring it into the repo as deploy/mvp-multinode/ so it's reproducible from a fresh clone. ## Why a separate driver? The single-host driver (deploy/scripts/run_mvp_demo.sh) is fine for PR-time smoke but flatters the bandwidth claims because every "edge" is loopback. The 4-node driver places the producers (node0/3), the gateway (node1), and the backend stack (node2) on separate boxes on a 10 Gbps LAN so wire-bytes per cut edge reflect real NIC traffic. ## Topology - node0 (10.10.1.1) — producers + agent-a (data source) - node1 (10.10.1.2) — gateway (ASAP arm only) - node2 (10.10.1.3) — backend stack (asap-query-backend, minio, thanos-*, prometheus, embedded controller) - node3 (10.10.1.4) — producers + agent-b (data source) ## Image refs updated for Phase 9 topology.env's IMAGES array dropped `asap/controller:dev` per #373: the controller binary now ships from asap/query-backend:dev (multi-bin build in Dockerfile.backend), so there are only 3 images to build+distribute, not 4. run_demo.sh's backend bring-up was already updated for this in an earlier session (no separate controller container; controller is in-process inside the asap-backend container). The 3 images that DO need building+distributing on each node: - asap/asap-otel:dev (build_asap_otel.sh from ASAPCollector root) - asap/fake-exporter:dev (docker build deploy/fake-exporter/) - asap/query-backend:dev (Dockerfile.backend with 4 build contexts) ## What's NOT committed - results/ and logs/ — per-run transient artifacts, gitignored. Same convention as the single-host driver (deploy/eval-results/ is also gitignored). Never commit historical reports — they get stale fast and look like source of truth. - autopilot.log — transient sweep log. - scripts/run_mvp_demo.sh from the source tree — was a stale rsync target; the canonical single-host driver lives at deploy/scripts/run_mvp_demo.sh and run_demo.sh rsyncs deploy/scripts/ to each node at bring-up. ## README deploy/mvp-multinode/README.md documents the topology, image set, bring-up commands (with the correct Dockerfile.backend invocation including the four build contexts), knobs, and the per-node artifact layout. Includes a comparison table single-host vs 4-node so people pick the right driver for the question they're asking. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * deploy/mvp-multinode: prune configs to only what run_demo.sh mounts run_demo.sh + topology.env reference exactly these 9 files; the other 40 were sweep / experiment / alt-storage configs from sibling work that crept into the multinode tree's configs/ during earlier sessions. Pruning so the PR is reviewable + the directory shows intent. Kept (mounted by run_demo.sh): - asap-otel-agent-b0-prometheus.yaml (baseline arm — b0) - asap-otel-agent-b1-serf-prometheus.yaml (b1 arm) - asap-otel-agent-b6-asap-single-sketch.yaml (asap arm) - asap-otel-gateway-mvp-placeholder.yaml (gateway, ASAP only) - backend-streaming.yaml (backend ingest) - backend-storage-routing.yaml (backend warm-tier + archive routing) - mvp-workload.yaml (controller workload spec) - prometheus-with-remote-write.yml (B0 Prometheus scrape config) - thanos-objstore.yaml (thanos sidecar/compact MinIO objstore) Removed: 40 unused YAML / JSON / md / test-data files. * cleanup: update single-host configs for Phase-9 single-binary refactor Two stale `asap/controller:dev` image references hung around in single-host topology after #373 retired the standalone controller image: - deploy/docker-compose/e2e-overlay.yml — overrode the controller service's image: + build:. With base.yml now pinning the right image (asap/query-backend:dev) and there being no build stanza to neutralize, both overrides are unnecessary; the overlay keeps its CONTROLLER_WORKLOADS + CONTROLLER_BACKEND_ENDPOINT env tweaks + workloads.yaml mount. - deploy/helm/asap/values.yaml — controller chart key updated to image: asap/query-backend:dev with entrypoint: /usr/local/bin/controller, mirroring the docker-compose pattern. Also: add an "Essential YAML configs the MVP demo touches" section to docs/mvp-demo-runbook.md so the 9-file working set is documented in one place (vs the 49 yaml files in deploy/configs/ that mostly belong to baseline-sweep / alt-storage overlays). Adds the single-binary image table for completeness. No deletions in deploy/configs/ — the env-var-parameterized mounts (e.g. ${AGENT_CONFIG:-...}) make it hard to prove orphan-ness via a static grep, and the baseline-sweep overlays exercise many of those files. The runbook now lists the essentials so readers don't have to guess. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(scripts): demo queries use raw metric names + distinct_over_time for HLL Two Phase-1-era staleness bugs in the replay/e2e query JSONs: 1. Per-sketch metric-name suffixes (_quantile, _kll, _hll) were dropped in #372 — the backend ingests under raw input names and the OTLP pdata variant tag carries the sketch type. The query JSONs still referenced the legacy suffixed names `http_requests_total_latency_ms_{quantile,kll,hll}`. Switched to the raw metric names per `mvp-workload.yaml`: - DDSketch quantile → http_requests_total_latency_ms - KLL quantile → http_requests_total_latency_ms - HLL cardinality → unique_users_per_min 2. The HLL JSON used `count_over_time(metric[r])` which counts SAMPLES, not distinct cardinality. Replaced with MetricsQL's `distinct_over_time(metric[r])` — the natural primitive for the HLL cardinality estimate. See https://docs.victoriametrics.com/victoriametrics/metricsql/#distinct_over_time CMS / CS JSONs already use the raw name `http_requests_total`; left unchanged. NOTE: the backend's warm-tier analyzer (warm_tier_analysis.rs in controller, PR #128) does not yet recognize `distinct_over_time` — it accepts `count_distinct_over_time` / `cardinality_estimate` / `count_distinct`. The analyzer-side alias landing is a follow-up that pairs naturally with the Prometheus → VictoriaMetrics swap; until that lands, the HLL queries here will surface as `UnsupportedFunction("distinct_over_time")` → archive forward, which is the safe fallback. Both `deploy/scripts/` (canonical) and `deploy/mvp-multinode/scripts/` (rsync source) updated to keep them in sync — the multinode orchestrator rsyncs deploy/scripts/ to each node at Phase 0 anyway, but the multinode copy is committed to make the 4-node demo reproducible from a fresh clone. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(scripts): rename 'promql' JSON field to 'metricsql' The replay queries now exercise MetricsQL (VictoriaMetrics superset of PromQL) — yesterday's HLL fix landed `distinct_over_time` which is MetricsQL-only. Keeping the JSON field named `"promql"` was misleading; rename to `"metricsql"` so the schema matches the language. ## Files - 5 query JSONs (canonical at deploy/scripts/ + rsync copies at deploy/mvp-multinode/scripts/): cms, cs, hll, kll, e2e. - promql_replay.py: `load_queries` now reads the `metricsql` key, with `"promql"` accepted as a back-compat fallback (a one-line `q.get("metricsql") or q.get("promql")` guard) so external workload JSONs in user repos don't hard-break mid-migration. Internal record emits use the new key throughout. Docstring example updated. The Python script's filename (`promql_replay.py`) is unchanged for now — that rename is a larger separate concern (touches all the scripts that invoke it). Field-only rename keeps this PR atomic. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(deploy/mvp-multinode): group configs by arm, drop duplicate scripts/, finish promql→metricsql rename PR #375 follow-ups: 1. deploy/mvp-multinode/scripts/ was a byte-identical copy of deploy/scripts/ and unused. run_demo.sh rsyncs deploy/scripts/ to each node at Phase 0 and invokes ${ROOT}/deploy/scripts/ for local analysis — the local mvp-multinode/scripts/ tree was dead weight + a drift hazard. Removed. 2. deploy/mvp-multinode/configs/ now groups by arm so each folder is self-contained for one baseline/asap arm: configs/b0/ B0 baseline agent configs/b1/ B1 baseline agent configs/asap/ ASAP agent + gateway + backend stack (streaming, storage-routing) + controller workload configs/shared/ Prometheus + Thanos (cross-arm) run_demo.sh mount paths and the file table in docs/mvp-demo-runbook.md updated. The Phase-0 rsync still pulls the whole configs/ tree recursively, so the subfolder structure propagates to all four nodes unchanged. 3. promql_replay.py → metricsql_replay.py (finishing the rename deferred in e0bbc35). Runtime contract was already MetricsQL- only after the HLL fix (distinct_over_time). All callers + docs updated (run_mvp_demo.sh, run_demo.sh, autopilot.sh, the two sweep scripts, plan_transition.py's import, measure-baseline's docstring, deploy/README.md, four doc pages). The load_queries() back-compat fallback for legacy "promql" JSON keys is preserved, with a regression test added. The inline replay-queries.json heredoc in run_mvp_demo.sh:536-542 that e0bbc35 missed is also migrated to "metricsql". Verification (static — multinode path requires 4 cloudlab nodes): - bash -n deploy/mvp-multinode/run_demo.sh: clean - every hardcoded /mydata/mvp-multinode/configs/... path in run_demo.sh resolves to an existing file under the new layout - python3 deploy/scripts/tests/test_metricsql_replay.py: 10/10 OK Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(deploy): split tree into mvp-singlenode/ + mvp-multinode/ per-demo packages PR #375 follow-up to the duplicate-scripts cleanup. The deploy/ tree mixed two demos in a single namespace: deploy/scripts/ did double-duty as both the single-host driver and the rsync source for the 4-node orchestrator, deploy/README.md described only the single-host path, and deploy/mvp-multinode/'s orchestrator scripts sat at the package root next to configs/. Reorganize so each demo is a self-contained package and the top-level deploy/ becomes an index. ## New layout deploy/ ├── README.md ← NEW: index pointing to both demos ├── docker/ ← universal Dockerfiles (unchanged) ├── fake-exporter/ ← Go source for producer image (unchanged) ├── helm/ ← K8s path (unchanged) ├── mvp-singlenode/ │ ├── README.md ← was deploy/README.md │ ├── scripts/ ← was deploy/scripts/ │ ├── configs/ ← was deploy/configs/ │ └── docker-compose/ ← was deploy/docker-compose/ └── mvp-multinode/ ├── README.md ├── topology.env ├── scripts/ ← *.sh moved here from package root │ ├── run_demo.sh │ ├── autopilot.sh │ ├── validate_arm.sh │ ├── snapshot_resources.sh │ ├── measure_freshness.sh │ └── measure_nic_bw.sh └── configs/ ← (already arm-grouped) ## Key non-obvious fixes - docker-compose YAMLs use `../configs/...` relative paths. Moving docker-compose/ AND configs/ together into singlenode/ keeps these working unchanged — no compose-file edits required. - mvp-multinode/scripts/{run_demo,autopilot}.sh now compute PKG_DIR="$(dirname ${SCRIPT_DIR})" and source topology.env from there (it stays at the package root, not under scripts/). - topology.env's CONFIG_SRC was pointing at deploy/configs/ (single-host) — pre-existing bug that round-2's per-arm subfolder reorg would have surfaced on first run. Repointed to deploy/mvp-multinode/configs/, which is what the per-arm mount paths actually expect. - deploy/mvp-singlenode/configs/tests/test_static_placeholder_5sketch_routing.py walks up to REPO_ROOT; bumped one more os.pardir + updated the embedded path string for the new layout. - 30+ docs/* and config-internal references rewritten via a scripted sed pass; verified zero `deploy/{scripts,configs,docker-compose}/` stragglers outside the singlenode/multinode trees. ## Verification (static) - bash -n: all 13 moved shell scripts clean - python3 test_metricsql_replay.py: 10/10 OK from new location - python3 test_static_placeholder_5sketch_routing.py: 9/9 OK - source mvp-multinode/topology.env: CONFIG_SRC and ROOT both resolve to existing directories - Every `../...` relative path inside the moved mvp-singlenode/docker-compose/*.yml resolves under the new tree (the one apparent miss is a `${GATEWAY_CONFIG:-...}` env-var substitution; resolved value exists) Multi-node runtime path can't be tested without a 4-node cloudlab allocation, but every static check that doesn't require ssh to peers passes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * rename(mvp-multinode): autopilot.sh → scripts/run_demo_sweep.sh Matches the singlenode side's run_*_sweep.sh convention (run-baseline-sweep.sh, run_e2e_sweep.sh, etc.) and pairs by prefix with the run_demo.sh it wraps. The PR description's own words for this script were "sweep wrapper" — the file name now agrees. Also updates: - Header docstring and `[autopilot]` log prefixes → `[sweep]` - README.md "Files" table: now shows scripts/ prefix on every driver, replaces the stale "(49 files)" claim on configs/ with the actual `{b0,b1,asap,shared}/` per-arm layout, and clarifies that the per-node Python utilities live under deploy/mvp-singlenode/scripts/ (rsync'd to each node at Phase 0) — the local mvp-multinode/scripts/ tree is only the orchestrator drivers. - `deploy/mvp-multinode/run_demo.sh` ref → `scripts/run_demo.sh` in the diff table - Removes the stale `autopilot.log` line from .gitignore; nothing writes to that path (run logs go under results/${RUN_ID}/run.log which is already covered by results/). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * cleanup(deploy/mvp-singlenode): delete 8 unused/superseded YAMLs Per a grep audit, all 8 files had zero load-bearing references — only stale doc comments and (for two of them) v3-compat alias mentions. The mvp-singlenode tree goes from 47 → 42 config YAMLs and 28 → 25 compose files. ## Deleted (no references anywhere) | File | Why dead | |---|---| | `configs/asap-otel-agent-b3-delta-e2e.yaml` | Header itself: "pipeline degenerates to nothing downstream of ddsketch flush" | | `configs/asap-otel-agent-passthrough.yaml` | Smoke-test relic; no compose / script / doc consumes it | | `configs/asap-otel-agent-gorillas3-tier.yaml` | Only mentioned in *comments* of other configs ("see pattern in gorillas3-tier"). Never mounted. | | `docker-compose/cross-host-parity.yml` | Phase 5 binary-mode parity harness — historical, no caller | | `docker-compose/trace-replay.yml` | CSV-trace-replay overlay; no caller (the fake-exporter has its own trace-replay mode, unrelated) | | `docker-compose/mvp-no-resource-limits.yml` | Override designed for one specific debugging session — no caller | ## Deleted (v3-compat aliases — v3 is gone) | File | Replaced by | |---|---| | `configs/asap-otel-agent-b1-serf.yaml` | `asap-otel-agent-b1-serf-prometheus.yaml` (v4 PRW variant, what `baseline-b1-serf.yml` already defaults to) | | `configs/asap-otel-agent-b5-gorilla.yaml` | `asap-otel-agent-b5-gorilla-prometheus.yaml` (v4 PRW variant, what `baseline-b5-gorilla.yml` already defaults to) | `baseline-b{1-serf,5-gorilla}.yml` carried v3-compat comment blocks saying "legacy callers using AGENT_CONFIG=…b1-serf.yaml still work" — those blocks are removed. The only remaining v3 caller was `run-baseline-sweep.sh:84,88` which hard-coded the v3 filenames; bumped to the -prometheus variants. ## Comment housekeeping for surviving files - `b1-serf-prometheus.yaml` and `b5-gorilla-prometheus.yaml` header comments dropped the "variant of X.yaml" cross-ref to the now-deleted v3 file. - `asap-otel-agent-kafka-fragment.yaml` and `asap-otel-gateway-kafka-fragment.yaml` had 4 comment lines pointing at gorillas3-tier as the canonical pattern reference; rephrased to "the older gorillas3-tier config (now removed)". - `docs/design-archive-tier.md:596` bumped to the v4 -prometheus filename. - `deploy/mvp-multinode/configs/b1/asap-otel-agent-b1-serf-prometheus.yaml` (the multinode duplicate) updated for the same dangling cross-ref. ## Verification - bash -n deploy/mvp-singlenode/scripts/run-baseline-sweep.sh: OK - grep audit: zero remaining references to the 8 deleted files (the 3 false-positive hits were unrelated: "trace-replay" the fake-exporter mode name, and "cross-host-parity" the `integration/cross-host-parity/` test directory) Note: the per-family matrix (4× backend-inference / 4× backend- streaming / 4× e2e-overlay / 4× asap-otel-agent-*-direct) was *not* touched — those are intentional per-family configs used by `run_e2e_sweep.sh:115-118` and documented as such in the README. The kafka cold-path demo trio (mvp-kafka-cold-path.yml + two kafka-fragment.yaml) was also kept — it's a manually- invoked feature demo, not dead code. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(deploy/mvp-singlenode): collapse 4 per-family e2e overlays into 1 parametrized file The four files `e2e-overlay-{cms,cs,hll,kll}.yml` were byte-identical apart from the `${FAMILY}` slug in the two volume mount paths: services: backend: volumes: - ../configs/backend-streaming-<FAM>.yaml:/etc/asap/streaming.yaml:ro - ../configs/backend-inference-<FAM>.yaml:/etc/asap/inference.yaml:ro Real duplication, unlike the per-family backend-streaming / backend-inference yamls (those encode genuinely different sketch-family schemas + query catalogs and were intentionally left alone). ## What changed - New `e2e-overlay-family.yml` (1 file) uses `${FAMILY}` substitution for both mounts. - Deleted `e2e-overlay-{cms,cs,hll,kll}.yml` (4 files). - `run_e2e_sweep.sh` SKETCHES array: per-family entries now name `e2e-overlay-family.yml` instead of `e2e-overlay-<fam>.yml`. Both `docker compose down` and `up` invocations now export `FAMILY="$FAM"` alongside the existing `AGENT_CONFIG="$AGENT_YAML"`. - README.md inference-overlay table: "Mounted by" column shows `e2e-overlay-family.yml (with FAMILY=cms)` etc. ## Verification - bash -n run_e2e_sweep.sh: OK - `FAMILY=cms docker compose -f base.yml -f e2e-overlay.yml -f e2e-overlay-family.yml config` resolves the volume sources to `backend-{streaming,inference}-cms.yaml` — interpolation works - grep audit: zero references to the 4 deleted family-overlay filenames anywhere docker-compose/ count: 25 → 22. Note on what's NOT collapsed: the 4× backend-{streaming,inference}-* yamls and the 4× asap-otel-agent-*-direct yamls are not byte-near- duplicates the same way. Their differences are per-family schemas (different sketch types, different parameter shapes, different metric naming) — templating those would just move divergence into conditional logic without reducing complexity. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(deploy/mvp-singlenode): swap Prometheus → VictoriaMetrics (Step 2g) Replaces every Prometheus container + config in the single-node demo with VictoriaMetrics. The multinode demo was already swapped to VM by an earlier commit on this branch. VictoriaMetrics is a Prometheus-API-compatible TSDB that also implements MetricsQL (a PromQL superset including `distinct_over_time` which yesterday's HLL JSON fix relies on). VM accepts remote_write + OTLP HTTP natively without special flags. ## Service renames | Was | Is | |---|---| | `prometheus` (base.yml, port 9090) | `victoriametrics` (port 8428 inside; 9090 on host) | | `prometheus-b0` (multi-stage, port 19090) | `victoriametrics-b0` (8428 inside; 19090 on host) | | `prometheus-archive` (archive overlay, 29090) | `victoriametrics-archive` (8428 inside; 29090 on host) | ## File renames - `configs/asap-otel-agent-b0-prometheus.yaml` → `…-b0-victoriametrics.yaml` - `configs/asap-otel-agent-b5-gorilla-prometheus.yaml` → `…-b5-gorilla-victoriametrics.yaml` - `docker-compose/mvp-prometheus-archive.yml` → `mvp-victoriametrics-archive.yml` (b1's config kept its filename — was already on VM in an earlier commit.) ## File deletions - `configs/prometheus.yml` - `configs/prometheus-with-remote-write.yml` - `configs/prometheus-otlp-receiver.yml` ## Key flag changes | Was (Prometheus) | Is (VictoriaMetrics) | |---|---| | `--web.enable-remote-write-receiver` | (built-in; no flag needed) | | `--web.enable-otlp-receiver` | (built-in; OTLP HTTP at `/opentelemetry/api/v1/push`) | | `--enable-feature=otlp-deltatocumulative` | (no equivalent needed — VM's OTLP receiver accepts both temporalities natively) | | `--storage.tsdb.retention.time=2h` | `-retentionPeriod=2h` | | `--storage.tsdb.path=/prometheus` | (default storage path; no flag) | | `--config.file=…/prometheus.yml` | (no config file required) | | `--web.listen-address=:9090` | `-httpListenAddr=:8428` | PRW endpoint at `/api/v1/write` is identical between Prometheus and VM — the `prometheusremotewrite` OTel exporter works against VM unchanged. Only the endpoint URL in the agent configs needed updating. ## Script changes - `run_mvp_demo.sh`: `HOST_PROM_B0_PORT` → `HOST_VM_B0_PORT` with back-compat shim: `HOST_VM_B0_PORT="${HOST_VM_B0_PORT:-${HOST_PROM_B0_PORT:-19090}}"` Plus log strings + service name refs. - `measure_freshness.py`, `mvp_report.py`, `measure-baseline.py`, `run-baseline-sweep.sh`, `run_freshness_phase.sh`: comment refresh for renamed configs. ## Grafana `grafana-datasources.yml` kept `type: prometheus` (Grafana's Prometheus datasource is wire-compatible with VM); changed `name:` to `VictoriaMetrics` and `url:` to `http://victoriametrics:8428`. ## Compose validation All `docker compose … config` invocations exit 0: - `base.yml` - `base.yml -f mvp-multi-stage.yml --profile b0` - `base.yml -f mvp-multi-stage.yml -f mvp-victoriametrics-archive.yml` - (bonus) asap-mode + thanos-archive - (bonus) baseline-{b0,b1,b5}.yml + agents-N10.yml ## Diff stat 23 files changed, +417 / -559 (net -142, three Prometheus YAMLs deleted) ## Open caveats - `mvp-freshness-probes.yaml`'s `target_path: prometheus_b0` field was left as-is — it's a semantic routing identifier, not a hostname. - `baseline-{b0,b1,b5}-*.yml` overlays had their `prometheus:` service overrides dropped — those injected Prometheus-only feature flags and remounted now-deleted config files. They are now label-only overlays; VM in base.yml already serves the PRW endpoint these baselines need. - `e2e-overlay.yml`: backend's `--prometheus-server` flag value changed from `http://prometheus:9090` to `http://victoriametrics:8428`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(deploy/mvp-singlenode): keep Prometheus as opt-in storage backend Yesterday's 2g commit made VictoriaMetrics the default storage tier for the single-node demo. Users may still want the Prometheus path (apples-to-apples baseline numbers vs the original architecture, or compatibility with externally-deployed Prometheus instances). Restore the Prometheus stack as an opt-in overlay alongside VM. ## Restored from history - `configs/prometheus.yml` (generic scrape) - `configs/prometheus-with-remote-write.yml` (b0 PRW receiver config) - `configs/prometheus-otlp-receiver.yml` (Mode-3 archive OTLP config) - `configs/asap-otel-agent-b0-prometheus.yaml` (b0 agent PRW → prometheus-b0:9090) - `configs/asap-otel-agent-b5-gorilla-prometheus.yaml` (b5 agent PRW → prometheus:9090) - `docker-compose/mvp-prometheus-archive.yml` (parallel to mvp-victoriametrics-archive.yml) ## New: `docker-compose/mvp-prometheus.yml` overlay When layered on top of `base.yml + mvp-multi-stage.yml`, this overlay: 1. Disables the VM services (`victoriametrics`, `victoriametrics-b0`) by reassigning their `profiles:` to `__disabled_when_prometheus_overlay__` via the `!override` Compose extension (default merge would have merged the arrays and left VM activated under --profile b0). 2. Re-targets grafana's `depends_on: victoriametrics` to point at prometheus instead, and swaps its datasource mount to a new `grafana-datasources-prometheus.yml`. 3. Defines `prometheus` (always-on, port 9090) + `prometheus-b0` (profile-gated under `b0`, port 19090) with command-line flags equivalent to what was there pre-VM-swap. ## New: `configs/grafana-datasources-prometheus.yml` Parallel grafana-datasources file pointing at `http://prometheus:9090` instead of `http://victoriametrics:8428`. Mounted by the overlay. ## Usage ```bash # Default (VictoriaMetrics): docker compose \ -f deploy/mvp-singlenode/docker-compose/base.yml \ -f deploy/mvp-singlenode/docker-compose/mvp-multi-stage.yml \ --profile b0 up -d # Prometheus alternative: AGENT_CONFIG_A=asap-otel-agent-b0-prometheus.yaml \ AGENT_CONFIG_B=asap-otel-agent-b0-prometheus.yaml \ docker compose \ -f deploy/mvp-singlenode/docker-compose/base.yml \ -f deploy/mvp-singlenode/docker-compose/mvp-multi-stage.yml \ -f deploy/mvp-singlenode/docker-compose/mvp-prometheus.yml \ --profile b0 up -d ``` For the asap-arm's archive tier, pair with `mvp-prometheus-archive.yml` instead of `mvp-victoriametrics-archive.yml`. ## Validation - `docker compose -f base.yml -f mvp-multi-stage.yml --profile b0 config` → ✓ valid (VM activated, no Prometheus) - `docker compose -f base.yml -f mvp-multi-stage.yml -f mvp-prometheus.yml --profile b0 config` → ✓ valid (Prometheus activated, VM profile-disabled) - Effective profile check: with overlay, `victoriametrics-b0 profiles=['__disabled_when_prometheus_overlay__']` (no longer in `b0`); `prometheus-b0 profiles=['b0']` Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: zz_y <zz_y@node0.zz-y-304941.softmeasure-pg0.clemson.cloudlab.us> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Q9 — Trade price index (NOP)
Adds ground truth scaffolding for Q9: trade price index computation.
Benchmark results
Q9: ❌ Not run yet — NOP collector; throughput benchmark pending.
Cumulative (Q1–Q8): Q1: 11/12⚠️ | Q3: not run | Q4: 12/12 ✅ | Q5: 3/12 ⚠️ | Q6: 12/12 ✅ | Q7: 12/12 ✅ | Q8: 0/4 ❌ (recall fails;
accuracy_sla=0.0001fix pending)See
results/benchmarking.mdfor full Q1–Q8 results.Made with Cursor