Skip to content

fix(monitor): coordinated sampling allocates on per-sid (agg-id) freq f_i, not total rate - #377

Merged
zzylol merged 1 commit into
mainfrom
fix/coordinator-per-key-freq
Jun 17, 2026
Merged

zzylol merged 1 commit into
mainfrom
fix/coordinator-per-key-freq

Conversation

@zzylol

@zzylol zzylol commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Problem

coordinator.rs::allocate_p() passed the edge rate vector as both the rate and the freq argument to allocate_sample_rates():

let p_vec = allocate_sample_rates(&rates, &rates, var_budget);

Since p_i = clamp(√λ·√(f_i/rate_i), 0, 1) and f_i == rate_i, every √(f_i/rate_i) = 1 → the allocation collapses to a uniform p. All edge differentiation was left to the ε-floor (which depends on rate only). So the headline p_i ∝ √(f_i/rate_i) coordinated-sampling claim was not actually exercised — the comment even said "freqs proxy = rates (no per-key split available at the coordinator)."

Fix

The MonitorReport already carries known_value — the monitored functional's per-edge value this epoch (the cms_point key frequency, or the edge's contribution to the monitored sum), which is the correct f_i and is distinct from the edge's total update rate. Feed it as the freq vector:

let freqs: Vec<f64> = ids.iter().map(|id| self.edges[*id].known_value).collect();
let p_vec = allocate_sample_rates(&rates, &freqs, var_budget);

Now an edge carrying more of the monitored key at equal total rate keeps a higher p; an edge with no monitored-key mass (known_value=0) falls back to p=1.

Tests

  • New per_key_freq_differentiates_at_equal_rate: equal rate, different known_value ⇒ different p, both above the equal rate-floor (fails on pre-fix code).
  • Existing skewed_rates_yield_differentiated_sample_p_above_floor still passes.
  • 13/13 monitor::coordinator tests green.

Found while wiring Fig 9 (coordinated sampling) on the 8-node eval cluster.

🤖 Generated with Claude Code

… f_i, not total rate

coordinator.rs::allocate_p() passed the edge `rate` vector as BOTH the rate and
the freq argument to allocate_sample_rates(), so √(f_i/rate_i)=1 for every edge
and the KKT allocation collapsed to a uniform p — all differentiation was left to
the ε-floor (rate-only). The MonitorReport already carries `known_value`: the
monitored functional's per-edge value, i.e. the frequency of the monitored series
id (the cms_point sid — internally the CMS `key`) / the edge's sum contribution,
identified by the report's aggregation_id. That, not the total update rate, is the
correct f_i. Feed it as the freq vector so the allocation is genuinely
p_i ∝ √(f_i/rate_i), evaluated per-aggregation per-edge: an edge carrying more of
the monitored sid at equal total rate now keeps a higher p. Edges with no
monitored-sid mass (known_value=0) fall back to p=1.

New test per_key_freq_differentiates_at_equal_rate pins the freq-driven branch
(equal rate, different known_value ⇒ different p, both above the equal rate-floor);
the existing skewed-rate (floor-driven) test still passes. 13/13 monitor tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@zzylol
zzylol force-pushed the fix/coordinator-per-key-freq branch from 140fb3d to 1beedb4 Compare June 16, 2026 20:53
@zzylol zzylol changed the title fix(monitor): coordinated sampling uses per-key freq, not total rate fix(monitor): coordinated sampling allocates on per-sid (agg-id) freq f_i, not total rate Jun 16, 2026
@zzylol
zzylol merged commit 160b858 into main Jun 17, 2026
@zzylol
zzylol deleted the fix/coordinator-per-key-freq branch July 17, 2026 20:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant