tune: fake-exporter cardinality down (5K aggregate target, was 25K+) - #352
Merged
Merged
Conversation
The five-sketch workload's HLL inner-label fan-out (user_id × outer label set) was the dominant SDK output contributor, pushing the post-PR-#338 demo to ~2.46 MB/s and blowing up agent → gateway bandwidth. Lower the two defaults that drive that fan-out: - EXPORTER_FIVE_SKETCH_USER_POOL: 1000 -> 100 (floor 500 -> 50). HLL still has a non-trivial active-user cardinality to estimate (100 distinct ids), and the rotate window keeps the per-minute semantic intact. - EXPORTER_CARDINALITY (binary default): 1000 -> 500 so direct-binary use also matches the demo script's PER_AGENT_CARDINALITY=500 default. With N_PRODUCERS=10 the aggregate gateway cardinality lands at ~5K series (was 10K-25K+). top_endpoint_qps / endpoint_request_freq stay at 50 endpoints (already small); request_size_bytes is scalar. PER_AGENT_CARDINALITY=500 in run_mvp_demo.sh and the mvp-multi-stage compose env propagate already match the new binary default; mvp-workload.yaml does not pin cardinality. Verified with go test ./deploy/fake-exporter/... and a local binary smoke (startup log shows cardinality=500 users=100 endpoints=50). Refs #46. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The post-PR-#338 five-sketch workload was emitting ~2.46 MB/s SDK output, blowing up agent -> gateway bandwidth and amplifying every other bug. The HLL inner-label fan-out (
user_idx outer label set) was the dominant contributor. This PR lowers two binary defaults so the demo lands at the documented ~5K aggregate-series gateway target.Changes (current -> new)
EXPORTER_FIVE_SKETCH_USER_POOLdeploy/fake-exporter/five_sketch_workload.goEXPORTER_CARDINALITYdeploy/fake-exporter/main.gorun_mvp_demo.sh'sPER_AGENT_CARDINALITY=500default;N_PRODUCERS=10 x 500 = 5Kaggregate.EXPORTER_FIVE_SKETCH_ENDPOINTSdeploy/fake-exporter/five_sketch_workload.gorequest_size_bytesAggregate gateway cardinality (math)
PER_AGENT_CARDINALITY=500outer label sets = 5,000 outer series (target).unique_users_per_min: ~5,000 outer x ~100 active user ids per rotation window, with rotation moving the active window through the pool — bounded inflation.request_size_bytes: scalar per outer = 5,000 series.Pre-tune we were at 10K-25K+ series; this brings the gateway closer to the runbook's 5K target line.
Driver / config sanity
deploy/scripts/run_mvp_demo.shline 111:PER_AGENT_CARDINALITY=500already matches.deploy/docker-compose/mvp-multi-stage.yml: per-producer env propagatesPER_AGENT_CARDINALITY:-500. No override above 500.deploy/configs/mvp-workload.yaml: no cardinality pin.EXPORTER_FIVE_SKETCH_USER_POOL/_ENDPOINTSare not set externally anywhere — the new binary defaults take effect.Don't / scope guard
request_size_bytes(KLL gauge),unique_users_per_min(HLL counter w/ user_id),top_endpoint_qps/endpoint_request_freq(CountSketch / CMS counter w/ endpoint).Test plan
go test ./deploy/fake-exporter/...— pass (all 4 metrics still emit; existing test uses explicitUSER_POOL=500so floor change is non-breaking).go build ./deploy/fake-exporter/...— pass.cardinality=500 users=100 endpoints=50 outer_card=500(wascardinality=1000 users=1000).Refs #46.
Generated with Claude Code (1M context).