Skip to content

feat(asap-precompute-go): telegraf codec (Phase 4 step B) - #234

Merged
zzylol merged 1 commit into
mainfrom
phase4/telegraf-codec
May 4, 2026
Merged

zzylol merged 1 commit into
mainfrom
phase4/telegraf-codec

Conversation

@zzylol

@zzylol zzylol commented May 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Phase B of the Telegraf integration plan in
docs/design-asap-telegraf-integration.md:
the asap-precompute-go/telegraf/ codec. Pure data-shape translation
between telegraf.Metric and the host-neutral
Observation / SketchEnvelope, mirroring asap-precompute-go/otel.

In scope

  • decode.go — Decode(metric, cfg) (*Observation, error). Reads
    cfg.ValueField as a float (KindFloat), or routes through
    cfg.EnvelopeField (default _asap_envelope_b64) as a
    pre-aggregated SketchEnvelope shortcut (KindEnvelope). Returns
    typed errors for missing / non-numeric value fields and malformed
    envelope payloads. Tags are sorted by key for stable series-key
    hashing — same contract as the OTel codec.
  • encode.go — Encode([]*SketchEnvelope, cfg) ([]telegraf.Metric, error).
    One emitted metric per envelope; env.MetricName as Name (with
    cfg.OutputMetricName fallback); env.ResourceLabels flattened into
    Tags alongside env.Labels (last-write semantics on collision so
    data-point Labels win); single field whose value is
    base64(JSON(SketchEnvelope)); timestamp from WindowEndMs
    (falling back to WindowStartMs, then time.Now()).
  • adapter.go — thin Adapter struct wrapping the package-level
    helpers; mirrors otel.Adapter shape but does NOT implement
    precompute.Adapter (no goroutines / tickers — those live in the
    Phase C plugin).
  • config.go — AdapterConfig with ValueField, EnvelopeField,
    OutputMetricName plus DefaultAdapterConfig().

Out of scope (deferred)

  • Phase C — the telegraf-patch/processors/allsketches/ plugin
    (StreamingProcessor, flush ticker, control-channel goroutine,
    config translation). Not touched in this PR.
  • Phase D — build_sketchtelegraf.sh + Telegraf submodule
    registration patch.
  • Phase E — cross-host envelope parity test against
    sketchcollector.
  • Multi-field metric fanout (single value_field only per design doc
    §10).
  • Custom asap Telegraf serializer (separate PR per design §4).

Test coverage

go test ./telegraf/... runs 22 tests covering:

  • Decode happy path: float scalar → KindFloat.
  • Decode int64 → cast to float.
  • Decode custom value field.
  • Decode missing value field → typed error.
  • Decode non-numeric value field → typed error.
  • Decode nil metric → error.
  • Decode envelope shortcut: base64-JSON envelope field → KindEnvelope,
    scalar field ignored.
  • Decode malformed envelope → wrapped error.
  • Decode with empty envelope-field config → shortcut disabled, falls
    back to scalar path.
  • Decode with nil config → uses package defaults.
  • Encode happy path: envelope → metric with base64-JSON field, tags,
    timestamp from WindowEndMs.
  • Encode metric-name fallback to cfg.OutputMetricName.
  • Encode flattens ResourceLabels into Telegraf tags.
  • Encode label / resource-label key collision: Label wins.
  • Encode nil envelope → indexed error.
  • Encode empty slice → nil slice, no error.
  • Encode empty EnvelopeField config → error (data-loss prevention).
  • Encode no timestamp → time.Now().
  • Encode WindowStartMs fallback when WindowEndMs is zero.
  • Encode field type-check (always plain string).
  • Adapter Decode / Encode wrap helpers.
  • Adapter constructor preserves config pointer.
  • Round-trip: Encode → Decode produces equivalent envelope (full
    field-by-field assertion).

go build ./..., go vet ./..., go test ./... all green; existing
otel/ and runtime tests unchanged.

Dependencies

Adds github.com/influxdata/telegraf v1.34.4 as a direct dep (pinned
to a release that matches the project's existing go 1.24.0
directive — newer 1.36+ telegraf releases require Go 1.25+, which
would force a toolchain bump that's out of scope here). Tidy left the
indirect-dep tree unchanged except for one zeebo/assert patch bump
that telegraf brought along — net diff is 1 require line in
go.mod and ~6 lines of go.sum. Telegraf does NOT pull in a heavy
transitive tree at this code-import depth (we only use
telegraf.Metric and metric.New).

Test plan

  • go build ./... from asap-precompute-go/
  • go vet ./...
  • go test ./telegraf/... (22 PASS)
  • go test ./... (all packages green; existing otel/ /
    controlchannel/ / sketches/ tests still pass)
  • Phase C will exercise the codec from a real Telegraf
    StreamingProcessor — that's where end-to-end correctness is
    proven; this PR alone doesn't gate on a sketchtelegraf binary
    build.

🤖 Generated with Claude Code

….Metric

Implements Phase B of the Telegraf integration plan in
docs/design-asap-telegraf-integration.md. The codec translates
telegraf.Metric ↔ Observation / SketchEnvelope, mirroring the OTel
codec in asap-precompute-go/otel.

Single-field decode (configurable ValueField); base64-envelope
shortcut for pre-aggregated upstream sketches. Encode emits the
envelope as a single base64-encoded field for binary-clean transport
through Telegraf's outputs.

NO plugin code — Phase C separately.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@zzylol
zzylol merged commit 12ed60c into main May 4, 2026
@zzylol
zzylol deleted the phase4/telegraf-codec branch May 4, 2026 15:10
zzylol added a commit that referenced this pull request May 5, 2026
Adds the Layer-4 OTAP-Rust codec at `asap-precompute-rs/src/otap/`,
gated behind the non-default `otap` Cargo feature. Mirrors the
two-layer split pinned by `docs/design-asap-otap-rust-integration.md`
§2 and the Phase-4 Telegraf codec (PR #234) in shape: pure data-shape
translation between `arrow::RecordBatch` and the host-neutral
`Observation` / `SketchEnvelope` types, no Tokio tasks, no timers,
no control-channel inbox. Lifecycle (Wakeup-driven flush, control
channel, linkme registration) is Phase C and explicitly deferred.

The Strategy-B carrier keys (_asap_envelope, _asap_sketch_type, etc.)
are defined as constants matching edge-framework §7.2's table
byte-for-byte so cross-platform interop holds.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant