gorilla-merger: Thanos-Receive-style fragment->TSDB merger (write path + StoreAPI)

[zzylol]

Summary

New co-located Go module gorilla-merger/ (own go.mod) implementing the Merger (Track 2) from .design_docs/GORILLA_MERGER_DESIGN.md (committed in this PR). It is a Thanos-Receive-style component, "Thanos Receive with an XOR-fragment frontend instead of remote-write."

End-to-end data flow:

• Ingest (POST /ingest/gorilla): read body, gunzip when Content-Encoding: gzip, gorilla.DecodeFragmentBatch (the shared wire codec from github.com/ProjectASAP/asap-gorilla-go), then per fragment chunkenc.FromData(chunkenc.EncXOR, f.Data), iterate the XOR chunk's samples, and appender.Append(labels, t, v). Labels = __name__=MetricName + fragment Attributes + the merger's external labels (external wins on conflict so distinct agents/mergers fan into distinguishable series). One appender per batch, committed once (WAL-durable on 200). OOO/duplicate samples are tolerated, not fatal. Malformed bodies -> 4xx; storage failures -> 5xx.
• Storage: embedded Prometheus tsdb.DB opened with the Prometheus default 2h Min/MaxBlockDuration and WAL on. The head holds the pending <2h window; it auto-compacts at the 2h boundary into on-disk blocks. Local retention kept short (default 6h) since blocks live in object storage once shipped.
• Ship: Thanos shipper.New(...) against a config-driven objstore bucket (the same bucket thanos-store-gateway watches), Sync on a ticker. One PUT set per 2h block (chunks + index + meta.json) with the Thanos thanos{} meta section written by the shipper.
• StoreAPI (the union mechanism): store.NewTSDBStore(logger, db, component.Receive, extLset) served over a gRPC storepb.StoreServer. This is the open-window query surface; register it in thanos-query's store list so it fans in alongside store-gateway, and thanos-query unions recent (<2h pending) + S3 (>=2h) data. The backend's existing ThanosQueryEngine (forward.rs) is unchanged.

The Rust backend is untouched.

Module layout

• cmd/gorilla-merger/main.go - env/flag wiring (HTTP addr, gRPC StoreAPI addr, tsdb dir, objstore config file, external labels, ship interval, retention). Shipper auto-disabled when no objstore config is supplied (write-path + StoreAPI still run).
• internal/merger/storage.go - tsdb.Open with 2h block range + WAL.
• internal/merger/ingest.go - the /ingest/gorilla handler + decode/append.
• internal/merger/storeapi.go - Thanos TSDBStore over gRPC.
• internal/merger/shipper.go - config-driven bucket + periodic shipper.Sync.

Dependency notes

• Pins github.com/prometheus/prometheus v0.308.0 and github.com/thanos-io/thanos v0.41.0 (mutually compatible; Thanos v0.41.0 itself requires prometheus v0.308.0). asap-gorilla-go's API surface I use (Encode/DecodeFragmentBatch, the Fragment type) carries no prometheus types, so the v0.307.3 -> v0.308.0 bump is transparent.
• asap-gorilla-go is pulled via a replace -> pinned checkout.
• Mirrors Thanos v0.41.0's own replace directives (vimeo/galaxycache -> thanos-community fork, the grpc fork, kuberesolver). Go does NOT inherit a dependency's replace directives into the main module, and the imported pkg/store -> cacheutil -> cache closure does not compile against upstream vimeo/galaxycache (its Codec interface differs). These mirrors are required for the build.

Test plan

• go build ./...
• go vet ./... clean, gofmt -l . clean
• go test ./... -count=1 green
• Ingest round-trip: XOR-encode known samples into gorilla.Fragments, gorilla.EncodeFragmentBatch, gzip, POST through the real httptest handler, query the tsdb.DB, assert samples + correct (external-merged) labels.
• Malformed body returns 400.
• Full write path: ingest -> force a 2h block cut (db.Compact) -> shipper Sync against an in-memory objstore bucket -> assert a complete block (chunks + index + meta.json) lands with the thanos{} section and the external label.

See .design_docs/GORILLA_MERGER_DESIGN.md for the topology, wire contract, and cost levers.

Generated with Claude Code.