feat(multinode): wire gorilla-merger into the ASAP arms (#32) - #421
Merged
Merged
Conversation
Run the asap-gorilla-merger (Thanos-Receive-style) on node2 in the ASAP arms: HTTP fragment ingest :10908, Thanos StoreAPI :10907 over the <2h pending window, 2h-block shipper into the asap-gorilla-tsdb bucket the store-gateway already watches. thanos-query gets a second --endpoint=gorilla-merger:10907 so it unions the merger's recent window with the store-gateway's >=2h S3 blocks (#32). Adds the gorilla-merger:NODE2_IP add-host alias, the data/gorilla-merger tsdb volume, and the asap/gorilla-merger:dev image to the distribution set. Documents the #24 gap: no edge agent path produces an HTTP ASAPFRG1 fragment ship today (the gorillas3 cold tier writes TSDB blocks straight to S3; there is no cold.ship_endpoint key), so the merger ingest port has no producer yet. The StoreAPI fan-out is live and queryable; end-to-end edge->merger->query validation is blocked on the producer side. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
asap-gorilla-merger(Thanos-Receive-style) on node2 in the ASAP arms of the mvp-multinode demo: HTTP fragment ingest:10908, Thanos StoreAPI:10907over the<2hpending window, 2h-block shipper into theasap-gorilla-tsdbbucket the store-gateway already watches (configs/shared/thanos-objstore.yaml). External labelscluster=asap-mvp,merger=m1.thanos-querygets a second--endpoint=gorilla-merger:10907alongside--endpoint=thanos-store-gateway:10901, so query unions the merger's recent<2hwindow with the store-gateway's>=2hS3 blocks (merger drops local blocks once shipped → no double-count).gorilla-merger:NODE2_IPadd-host alias, thedata/gorilla-mergertsdb volume dir, andasap/gorilla-merger:devto the image distribution set + README image table.#24 gap (documented, not fabricated)
There is no
cold.ship_endpointkey anywhere in this repo, and no edge agent path produces an HTTPASAPFRG1fragment ship today. The current edge cold tier is thegorillas3OTel processor writing Prometheus TSDB blocks directly to MinIO (block_format: prometheus_tsdb,tsdb_bucket: asap-gorilla-tsdb); itsagentrole emits fragment metrics downstream through the OTel pipeline, not an HTTP POST. So the merger's ingest port has no producer in the deploy yet. The README spells out the two options to close #24 (anASAPFRG1HTTP exporter, or agateway_fragmentsidecar that re-POSTs). The StoreAPI fan-out side is live and queryable; end-to-end edge→merger→query validation is blocked on the producer side only.Notes
asap/gorilla-merger:devis built by/mydata/asap-build-all.sh(host script, updated locally — not in this repo) fromASAPQuery-backend/gorilla-merger/Dockerfile(PR build(gorilla-merger): containerize via BuildKit secret (unblocks #310 CI) ASAPQuery-backend#312).bash -nclean onrun_demo.sh+topology.env.Test plan
asap/gorilla-merger:dev(needs thegh_tokenBuildKit secret) anddocker save | ssh loadit to the nodes.run_demo.sh up asap→ confirmasap-gorilla-mergeris up on node2, StoreAPI reachable, andthanos-querylists it as a healthy store (/api/v1/stores).POST /ingest/gorillafrom an edge producer and confirm the samples surface throughthanos-query.🤖 Generated with Claude Code