feat(deploy): three-axis sweep driver - #193
Merged
Merged
Conversation
New sweep driver that iterates the `(W, L, agg_type)` grid defined
in docs/sdk-aggregation-three-axis-design.md, bringing up the
compose stack once per cell, soaking, calling measure-baseline.py,
and tearing down.
Each of the paper's §6.2 sub-sweeps is a thin wrapper fixing two
axes and varying the third:
6.2a time: WINDOWS="1s 15s 60s 300s" PROJECTIONS=":" AGGS="dd-full"
6.2b label: WINDOWS="60s" PROJECTIONS=": zone,rack zone -"
AGGS="dd-full"
6.2c encoding: WINDOWS="60s" PROJECTIONS="zone,rack"
AGGS="raw-buffer dd-full dd-delta kll cms-full hll-full"
Implementation notes:
- PROJECTIONS uses ":" as a bash-friendly escape for "keep all
labels" (the natural empty-string would be awkward to pass
through a space-separated list) and "-" for "drop all".
decode_projection() maps those back to what
fake-exporter's EXPORTER_SDK_PROJECTION expects.
- Uses baseline-b0a-raw-stream.yml as a neutral agent shape
(OTLP → batch(1s) → OTLP), which keeps the agent from running
a sketch pipeline on top and conflating the SDK-side
measurement. The SDK axis is the one under study.
- Row tag is "w${W}-l${PROJ_RAW}-a${AGG}" so the resulting CSV
is trivially splittable back into the grid for plotting.
Smoke-tested locally against the PR #189/#190/#191 stack:
$ WINDOWS=5s PROJECTIONS=":" AGGS=raw-buffer SOAK_S=30 \
CARDINALITY=50 FREQ_HZ=5 BYTES_WIN=3 \
./deploy/scripts/run-three-axis-sweep.sh > /tmp/sweep.csv
baseline,scale,rate,cardinality,producer_cpu_cores,...
w5s-l:-araw-buffer,N1,,50,0.025,9.754,322.269,...
Driver brings up/down correctly, producer-side columns populate,
CSV is well-formed. Numeric validation of the aggregator output
(e.g., "does raw-buffer really emit every sample under gzip
compression?") belongs to the §6.2c sub-sweep PR that uses this
driver to collect actual data.
Stacked on feat/measure-baseline-producer-columns (#191) — that
PR adds the producer columns this script relies on.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zzylol
added a commit
that referenced
this pull request
Apr 23, 2026
…iner (#194) base.yml was hard-coding EXPORTER_RATE=1000 and ignoring the new EXPORTER_SDK_* knobs introduced in PR #190. Smoke-testing run-three-axis-sweep.sh (#193) produced suspiciously-low producer_bytes_out_per_s values because every cell of the sweep was silently running the SDK at its defaults (window=15s, agg=default, projection="") instead of the swept values. Switches the fake-exporter container's environment block to `${VAR:-default}` interpolation for every sweep-relevant knob so the sweep driver's env assignments actually reach the container: EXPORTER_FREQ_HZ EXPORTER_SDK_WINDOW EXPORTER_SDK_PROJECTION EXPORTER_SDK_AGG EXPORTER_MAX_BUFFER_PER_SERIES Legacy EXPORTER_RATE is kept but drops to empty by default so the deprecation warning doesn't fire spuriously. Before/after on the same cell (W=5s, card=50, freq=5, agg=raw-buffer, no gzip, 60s soak): before after sdk config logged default/15s raw-buffer/5s producer_bytes_out 1923 B/s 48546 B/s (×25) gateway_points_per_s 3.18 280.4 backend_samples_per_s 2.91 247.9 48 KB/s matches the naive estimate (50 card × 5 Hz × 2 instruments × ~80 B/datapoint ≈ 40 KB/s), confirming AggregationRawBuffer is emitting the expected per-event stream. Before the fix, the SDK was running Sum aggregation on the Counter instead — one data point per attribute set per 15s tick = ~7 pts/s, which matches the low gateway rate seen. Also adds `deploy/fake-exporter/sdk_emit_test.go` as a regression test for the aggregator contract: a ManualReader + AggregationRawBuffer should emit exactly `cardinality × instruments × samples_each` data points. This is the counterpart of the unit test in opentelemetry-go-patch/sdk/metric/internal/aggregate/ but runs at the MeterProvider+View level, catching wiring mistakes like the one above. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
(Re-opened from #192 which GitHub auto-closed when its base branch was squash-merged.)
Summary
New
run-three-axis-sweep.shthat iterates the(W, L, agg_type)grid fromdocs/sdk-aggregation-three-axis-design.md. Each paper §6.2 sub-sweep is a thin wrapper fixing two axes and varying the third:WINDOWSPROJECTIONSAGGS1s 15s 60s 300s:dd-full60s: zone,rack zone -dd-full60szone,rackraw-buffer dd-full dd-delta kll cms-full hll-full:= keep all labels (bash-friendly escape for empty string);-= drop all.How it works
docker compose down→upwithEXPORTER_SDK_WINDOW/PROJECTION/AGGset in env.baseline-b0a-raw-stream.ymlas the agent shape (OTLP → batch(1s) → OTLP) — neutral, keeps the agent from doing sketch work on top.SOAK_S(default 180s), then invokesmeasure-baseline.pyfor one CSV row.w${W}-l${PROJ_RAW}-a${AGG}.Verified locally
Not in this PR
Numeric validation of the aggregator shapes (is
raw-bufferreally emitting every sample? is gzip compressing everything to ~0?) belongs to the §6.2c sub-sweep PR that uses this driver.🤖 Generated with Claude Code