Skip to content

feat(multinode): wire gorilla-merger into the ASAP arms (#32) - #421

Merged
zzylol merged 1 commit into
mainfrom
feat/deploy-merger
May 23, 2026
Merged

zzylol merged 1 commit into
mainfrom
feat/deploy-merger

Conversation

@zzylol

@zzylol zzylol commented May 23, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Runs asap-gorilla-merger (Thanos-Receive-style) on node2 in the ASAP arms of the mvp-multinode demo: HTTP fragment ingest :10908, Thanos StoreAPI :10907 over the <2h pending window, 2h-block shipper into the asap-gorilla-tsdb bucket the store-gateway already watches (configs/shared/thanos-objstore.yaml). External labels cluster=asap-mvp,merger=m1.
  • Add batch/window modes, support sketch/raw SDK collection and benchmarking #32 — StoreAPI fan-out: thanos-query gets a second --endpoint=gorilla-merger:10907 alongside --endpoint=thanos-store-gateway:10901, so query unions the merger's recent <2h window with the store-gateway's >=2h S3 blocks (merger drops local blocks once shipped → no double-count).
  • Adds the gorilla-merger:NODE2_IP add-host alias, the data/gorilla-merger tsdb volume dir, and asap/gorilla-merger:dev to the image distribution set + README image table.

#24 gap (documented, not fabricated)

There is no cold.ship_endpoint key anywhere in this repo, and no edge agent path produces an HTTP ASAPFRG1 fragment ship today. The current edge cold tier is the gorillas3 OTel processor writing Prometheus TSDB blocks directly to MinIO (block_format: prometheus_tsdb, tsdb_bucket: asap-gorilla-tsdb); its agent role emits fragment metrics downstream through the OTel pipeline, not an HTTP POST. So the merger's ingest port has no producer in the deploy yet. The README spells out the two options to close #24 (an ASAPFRG1 HTTP exporter, or a gateway_fragment sidecar that re-POSTs). The StoreAPI fan-out side is live and queryable; end-to-end edge→merger→query validation is blocked on the producer side only.

Notes

Test plan

  • Build asap/gorilla-merger:dev (needs the gh_token BuildKit secret) and docker save | ssh load it to the nodes.
  • run_demo.sh up asap → confirm asap-gorilla-merger is up on node2, StoreAPI reachable, and thanos-query lists it as a healthy store (/api/v1/stores).
  • (Blocked on CountminSKetch Tumbling Window #24) exercise POST /ingest/gorilla from an edge producer and confirm the samples surface through thanos-query.

🤖 Generated with Claude Code

Run the asap-gorilla-merger (Thanos-Receive-style) on node2 in the ASAP
arms: HTTP fragment ingest :10908, Thanos StoreAPI :10907 over the <2h
pending window, 2h-block shipper into the asap-gorilla-tsdb bucket the
store-gateway already watches. thanos-query gets a second
--endpoint=gorilla-merger:10907 so it unions the merger's recent window
with the store-gateway's >=2h S3 blocks (#32). Adds the
gorilla-merger:NODE2_IP add-host alias, the data/gorilla-merger tsdb
volume, and the asap/gorilla-merger:dev image to the distribution set.

Documents the #24 gap: no edge agent path produces an HTTP ASAPFRG1
fragment ship today (the gorillas3 cold tier writes TSDB blocks straight
to S3; there is no cold.ship_endpoint key), so the merger ingest port has
no producer yet. The StoreAPI fan-out is live and queryable; end-to-end
edge->merger->query validation is blocked on the producer side.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@zzylol
zzylol merged commit e2f210f into main May 23, 2026
@zzylol
zzylol deleted the feat/deploy-merger branch May 26, 2026 11:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant