docs: record byte-parity completion + five-claim eval framing - #255
Merged
Merged
Conversation
Two updates rolled into one PR: ## PROGRESS.md — byte-parity completion (closes #243 follow-on) Adds a top-of-file dated section "Cross-language byte-format parity, 5/5 sketches (2026-05-05)" covering the chain of work that closed #243: - asap_sketchlib PRs #43 / #44 / #45 — HLL hash-seed alignment, CountSketch HashSpec / derive_index / derive_sign port (also extracts the shared `common::hashspec` module), CountMinSketch port routing through the same primitives. - ASAPCollector PR #254 — un-ignores `cms_byte_parity_with_go` with the matching FO-mode wrapper rewrite, and fixes a latent inconsistency in the HLL fixture generator that #252 missed (HLL was using `SerializeProtoBytes()` directly while every other sketch goes through `SerializePortable*` + strip + Marshal; the test only "passed" because fixtures are gitignored and rarely regenerated alongside a test run). - Verification: 7/7 cross_language_parity tests pass with `--include-ignored`, including all 5 byte-parity tests + 2 sanity tests, no ignored remaining. Also flags the downstream unblock: `ASAPQuery-backend`'s `edge_runtime_consumes_precompute_rs.rs` HLL / CS / CMS round-trip tests were `#[ignore = "blocked on ASAPCollector#243"]` per `design-phase3-asap-precompute-rs.md` — they should now pass without backend code changes (mechanical un-ignore PR pending, separate work). ## docs/paper-outline.md — five-claim eval framing The "Measurable benefits at every layer" bullet was 4 lines listing benefits without pointing at *how* the paper proves each one. Replaces it with five named evaluation dimensions (transmission bandwidth, edge CPU, edge memory, query accuracy, query latency) plus the combined Pareto headline claim, with explicit evidence-tooling pointers per claim: - bandwidth: `run_e2e_sweep.sh` (P7) bytes columns + `e2e_plots.py` (P9) bandwidth-vs-N + `cardinality_crossover/` single-host pre-compare. - edge CPU: P7 producer-side cpu column + `bench_2node_sim.sh` + the SDK label-axis profile (paper blocker #2). - edge memory: P7 rss column + `bench_soak.sh` (steady-state RSS / heap / fd-count + slope-based leak verdict). - accuracy: `raw_tee.go` (P4) ground truth + `accuracy_reduce.py` (P8) join + `ASAPQuery-backend/TODO.md` "Accuracy-profile library per sketch type" + `sketchlib-bench/docs/DESIGN.md` for bound derivation. - query latency: `promql_replay.py` (P5) p50/p99 + P9 `query_latency_cdf.png` + `plan_transition.py` (P6) transition timings. The combined Pareto claim (`pareto_acc_vs_thru.png` from P9) is named explicitly as the figure the paper's contribution rests on. The "Formal correctness" bullet stays unchanged. Architectural sibling — the existing [`design-asap-edge-framework.md`](design-asap-edge-framework.md) — covers the system shape; this rewrite makes the empirical bar for "done" explicit alongside it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two doc updates rolled into one PR.
PROGRESS.md — byte-parity completion
New top-of-file dated section "Cross-language byte-format parity, 5/5 sketches (2026-05-05)" covering the chain that closed #243:
common::hashspec), CountMinSketch port reusing those primitives.cms_byte_parity_with_gowith FO-mode wrapper, plus fixes a latent HLL fixture-generator inconsistency phase3(parity): un-ignore hll byte-parity test #252 missed (HLL was usingSerializeProtoBytes()directly while every other sketch goes throughSerializePortable*+ strip + Marshal).cross_language_paritytests pass with--include-ignored.Flags the downstream unblock for
ASAPQuery-backend'sedge_runtime_consumes_precompute_rs.rs(HLL / CS / CMS round-trip tests were gated on #243 perdesign-phase3-asap-precompute-rs.md).docs/paper-outline.md — five-claim eval framing
Replaces the 4-line "Measurable benefits at every layer" bullet with five named evaluation dimensions + each claim's evidence tooling, plus the combined Pareto headline claim explicitly named:
cardinality_crossover/bench_2node_sim.shbench_soak.shraw_tee.go(P4) +accuracy_reduce.py(P8) +sketchlib-bench/docs/DESIGN.mdpromql_replay.py(P5) + P9 query_latency_cdf +plan_transition.py(P6)Combined Pareto headline:
pareto_acc_vs_thru.png(P9) — the figure the paper's contribution rests on.Rationale: the original bullet listed benefits without pointing at how the paper proves each one. This makes the empirical bar for "done" explicit, complementary to the architectural framing in
design-asap-edge-framework.md.🤖 Generated with Claude Code