Skip to content

feat: coldpart Part format in asap-gorilla-go + agent intchunk cold-part ship path - #439

Merged
zzylol merged 1 commit into
mainfrom
feat/coldpart-relocate-and-agent
May 25, 2026
Merged

zzylol merged 1 commit into
mainfrom
feat/coldpart-relocate-and-agent

Conversation

@zzylol

@zzylol zzylol commented May 25, 2026

Copy link
Copy Markdown
Contributor

Summary

First of a two-PR change (merge this one first; the ASAPQuery-backend re-import PR follows).

  • Relocate coldpart to the shared lib. Adds asap-gorilla-go/coldpart (full read+write code + tests), next to intchunk, so BOTH the edge agent (producer) and the backend gorilla-merger (consumer) can import the cold "Part" format. The merger's internal/coldpart is deleted in the follow-up backend PR.
  • Agent intchunk cold-part path (opt-in, default OFF). Adds a parallel cold ship path in the asap_edge processor. When cold.format: intchunk (default is the existing gorilla-XOR fragment path, unchanged), the SAME per-window drained samples are re-encoded as a lossless coldpart.Part with absolute-ms block bounds and POSTed to the merger's POST /ingest/coldpart (cold.coldpart_endpoint). The merger derives the content-addressed key itself; the agent sends only the bytes.
  • Shared helpers. New exported gorilla.FragmentLabels (single source of truth for fragment→labels, also used by the fragment→TSDB finalizer) guarantees a series is queried under one identity regardless of cold format; gorilla.DecodeFragmentSamples recovers raw samples from the same drained XOR fragments.

The flag is wired through ColdConfig (the same config path that selects the current cold ship behavior); an operator typically sets it from an env var, e.g. cold.format: ${env:ASAP_COLD_FORMAT} with ASAP_COLD_FORMAT=intchunk.

Test plan

  • asap-gorilla-go: go build ./..., go test ./... (incl. moved coldpart tests), go vet, gofmt all clean.
  • asapedgeprocessor: builds + full go test ./... green; go vet/gofmt clean.
  • New integration tests: agent builds a part → coldpart.OpenPart + Series() round-trips samples bit-exactly with absolute-ms block bounds; POST to an in-process /ingest/coldpart-style handler validates via OpenPart; full ConsumeMetricsflushAll exercises the opt-in path end to end.
  • Default-off proof: with cold.format unset the processor keeps the fragment path and wires no cold-part shipper (no behavior change).

🤖 Generated with Claude Code

…-part ship path

Move the cold "Part" format (intchunk-coded read/write library) into
asap-gorilla-go/coldpart so both the edge agent (producer) and the backend
gorilla-merger (consumer) can import it; it previously lived in the merger's
internal package, which the agent cannot import.

Add an opt-in intchunk cold-part path to the asap_edge processor, parallel to
the default gorilla-XOR fragment ship path (unchanged when the flag is unset):
when cold.format=intchunk the same per-window drained samples are re-encoded as
a lossless coldpart.Part with absolute-ms block bounds and POSTed to the
merger's /ingest/coldpart (cold.coldpart_endpoint). Series labels are built via
the new exported gorilla.FragmentLabels so either format archives a series under
one identity; gorilla.DecodeFragmentSamples recovers raw samples from the shared
fragments.

This is the first of a two-repo change. A follow-up ASAPQuery-backend PR
re-imports coldpart from asap-gorilla-go and deletes its internal copy; merge
this PR first.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant