receipt: publish every duelled challenger's diagnostics (cohort_stats) - #245
Merged
Conversation
The signed receipt already carries every cohort challenger's full per-window
scores and its LCB under alpha/k, but the summary diagnostics — observed
geomean, window win rate, Wilcoxon p, bootstrap p50/p95, per-domain win
rates — exist only for the decided challenger (the headline verdict). A
dashboard or miner reading "how did MY entry do" for a non-crowned challenger
had to re-run the bootstrap.
VerdictRecord gains `cohort_stats: {challenger_hotkey: {geomean, win_rate,
wilcoxon_p, boot_p50, boot_p95, per_domain_win_rate}}`, filled by the
validator from the judged cohort next to `cohort_lcbs`. Same drop-when-default
rule: absent from the canonical body on single-challenger rounds and on every
archived receipt, so their signatures verify unchanged (checked against two
real receipts, one cohort and one pre-cohort). RECEIPT_VERSION stays 3.
`cascade-audit`'s duel-cohort check replays the published stats from the
recorded scores: a number that does not reproduce, or stats for a hotkey that
was never scored, fail the round; a receipt without the field passes as before.
Reader note for deployment: verification re-serialises the parsed receipt, so
a reader on older code drops the new field and fails the signature check on
new cohort receipts (skips them, no crash). Ship readers — trainer and
external validators' bootstrap path — before the validator starts writing it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ewrx7YTJ46UosQFgLfDhPf
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
tensorlink-dev
added a commit
that referenced
this pull request
Sep 6, 2026
Both sides add per-challenger receipt fields next to cohort_lcbs; kept both: cohort_stats (#245) alongside cohort_geomeans / cohort_per_horizon (#243). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ewrx7YTJ46UosQFgLfDhPf
tensorlink-dev
added a commit
that referenced
this pull request
Sep 7, 2026
main's cohort_stats PR (#245) added tests calling check_duel_cohort(receipt); DEC-CA-0038 gave it a required cfg arg. Adapt the 4 call sites. Co-Authored-By: Claude Code <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Q9Qtvsw3SjMTWjkpYqRNaT
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The signed receipt already records every cohort challenger's full per-window scores (
entry_scores) and its LCB under alpha/k (verdict.cohort_lcbs). The summary diagnostics — observed geomean, window win rate, Wilcoxon p, bootstrap p50/p95, per-domain win rates — were only recorded for the decided challenger. This addsverdict.cohort_stats: the same set for every duelled challenger, keyed by hotkey.Why
A dashboard or a miner asking "how did my entry do" on a cohort round could only answer for the crowned challenger without re-running the bootstrap. With 17 seated finalists tonight that gap is most of the field.
Compatibility (the part that matters)
cohort_lcbs: absent from the canonical body on single-challenger rounds and on every archived receipt.RECEIPT_VERSIONstays 3. Verified against two real receipts (round 11362022463128982272, k=3, and a pre-cohort one): canonical body and signature unchanged under the new parser.verify_receipt_signaturere-serialises the parsed receipt; a reader on older code drops the unknown field and the signature check fails on new cohort receipts. Affected readers: the trainer's dethrone probe (falls back to the sticky/incentive king, so the reign clock lags), and validators' throne bootstrap on restart (skips the receipt, next candidate). No consensus or weight path reads receipts. Order: release, external validators + trainer upgrade, then our validator.cascade-auditduel-cohortreplays the published stats from the recorded scores. Missing field ⇒ pass as before; a number that does not reproduce, missing stats for a duelled challenger, or stats for a never-scored hotkey ⇒ FAIL.Not changed
Scoring, gating,
KothParams, chain.toml, contract digest, state files, dashboard.Tests
cohort_stats; audit rejects a doctored geomean and stats for an unscored hotkey; audit accepts a legacy receipt without the fieldtests/unit: 1791 passed, 10 skipped; ruff clean🤖 Generated with Claude Code
https://claude.ai/code/session_01Ewrx7YTJ46UosQFgLfDhPf