Skip to content

specs(tools/tri): real tests for 11 tool cards, gen to merging (Closes #7903) - #7904

Merged
gHashTag merged 3 commits into
masterfrom
vac/tools-tri-1
Oct 8, 2026
Merged

specs(tools/tri): real tests for 11 tool cards, gen to merging (Closes #7903)#7904
gHashTag merged 3 commits into
masterfrom
vac/tools-tri-1

Conversation

@gHashTag

@gHashTag gHashTag commented Oct 8, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Closes #7903. Part of #6063 (t27b coverage), which is item C10 of #6488.

What changes

Eleven tool cards in specs/tools/tri had no test block. They passed vacuously (0 runtime asserts) under the reference and under t27b, and broke L4 TESTABILITY. Each now carries the same four tests, which check the rules the card is written under. The rules are specs/tools/catalog.t27 (ID_RULE, SCHEMA_2_RULE) and the card's own header (the tri <name> command, read off a clap #[derive(Subcommand)] enum).

test what it checks
qualified_id_is_repo_colon_id QUALIFIED_ID is REPO, a :, then ID, byte by byte; SCHEMA is 2
id_command_and_variant_name_one_subcommand ID is tri/<name>, COMMAND is tri <name>, and VARIANT is Commands::<Name>, where <name> is the kebab-case of <Name> (OneAway is one-away)
clap_names_are_kebab_case the command name and every action are lower-case words joined by -; one ACTIONS_ABOUT per action; every ARGS entry is a <positional> or a --flag
the_queen_holds_every_tool_beside_its_lead AGENTS has the Queen (T) first, as docs/agents/AGENTS_ALPHABET.md binds every tool to her, then the domain lead(s): one letter each, none twice

No test restates a constant: each one relates two or more fields, or a field to the grammar it must follow. Each test walks strings with a var index, so its asserts are runtime asserts.

Specs changed (all specs/tools/tri/<name>.t27), with the runtime asserts their 4 tests execute (reference / t27b):

spec reference t27b
gen 4/4 4/4, 53 asserts
harness 4/4 4/4, 76 asserts
health 4/4 4/4, 65 asserts
hooks 4/4 4/4, 148 asserts
issues 4/4 4/4, 88 asserts
jumps 4/4 4/4, 67 asserts
kinds 4/4 4/4, 66 asserts
lean 4/4 4/4, 70 asserts
ledgers 4/4 4/4, 74 asserts
loop 4/4 4/4, 62 asserts
merging 4/4 4/4, 70 asserts
  • Reference: t27c test-report <spec> on the t27c lab (zig 0.16.0): 4 pass, 0 fail, 0 vacuous for each.
  • t27b: t27b test <spec> --check on the t27b lab (aarch64 under qemu).
  • Seals: t27c seal --save, then t27c seal --verify: all hashes MATCH. The 11 seal files are new; these cards had none.
  • No existing line of any spec changed; the tests are appended.

Mutants

13 hand mutants per card, each a plausible slip in the data. Each was applied to a copy of the card and run under t27c test-report:

  • QUALIFIED_ID names the other repository, or its name drifts from ID;
  • COMMAND joined by -, or its name drifts;
  • VARIANT with the enum renamed, the name drifted, or written in lower case;
  • ID loses its /;
  • AGENTS with the lead before the Queen, or the Queen twice;
  • the first action not kebab-case;
  • the first arg neither <...> nor --...;
  • SCHEMA 1.

Over the 31 cards of specs/tools/tri, 359 mutants applied and 359 were killed. The other 31 did not apply: an action or arg mutant on a card with no actions or no args. None survived.

Skipped

None in this directory: all 31 cards of specs/tools/tri get these tests, in this PR and two more (misread to red, reseal to worktrees). The specs this lane leaves alone elsewhere (module-path fixtures such as specs/a/b_c.t27, and empty placeholder modules) are named in the PRs of their directories.

Gates

On the t27c lab, from master's gen/c/policy/own_language.c, with lefthook.yml's one-line C main:

  • check_all() over origin/master:tools/policy/foreign-exceptions.txt, -- and git diff --name-status origin/master...HEAD: exit 0. A planted A cli/foo/new.rs line exits 1.
  • check_budget() over git diff --numstat --no-renames origin/master...HEAD: exit 0. No foreign line is added.

Ledger

docs/reports/t27b_expectations.json: the 11 rows move from pass_vacuous to pass; nothing else changes. The rows come from bless in scripts/tri_loop/t27b.py (what tri t27b ratchet --bless runs), composed onto master's ledger with only these rows. After master 408d740 changed the ledger, master was merged in from master's side (2e8539e) and the 11 rows were recomposed onto its ledger: pass 931 -> 942, pass_vacuous 151 -> 140, max_not_pass unchanged.

tri t27b ratchet --run of this branch's lab run (below) against the branch ledger has no finding on any of the 11 specs. Against master's ledger the same run gives 11 UNEXPECTED PASS (ledger pass_vacuous, now a pass with N runtime asserts). The run's other findings (2 UNEXPECTED PASS, 73 UNLISTED, 1 STALE, 4 MOVED) are master's own and are not touched here.

t27b lab: signed receipts (#7686)

Head 967bb7b (this branch), requested with a fresh 32-byte challenge, against base 76836e2. The base is the newest master ancestor the lab has a receipt for; the merge-base 8378792 has none. Both are signed runs of the deployed lab (rustc 1.99.0, zig 0.16.0, qemu-aarch64 7.2).

/work/t27c-master corpus-receipt compare BASE HEAD --challenge-head <mine>, from /work/t27 on the t27b lab:

base "76836e23e4580a3fb3c87bf5ef6bfd3f713a2c7a" AUTH_MISSING_NONE AUTHOR leaves-bound true
head "967bb7b202fb2637c14507d7a3094f1260f84c5b" AUTH_MISSING_NONE FRESH leaves-bound true
  lane neutral specs/compiler/core/t27core.t27
  lane neutral specs/queen/actors.t27
  lane improved specs/tools/tri/gen.t27
  lane improved specs/tools/tri/harness.t27
  lane improved specs/tools/tri/health.t27
  lane improved specs/tools/tri/hooks.t27
  lane improved specs/tools/tri/issues.t27
  lane improved specs/tools/tri/jumps.t27
  lane improved specs/tools/tri/kinds.t27
  lane improved specs/tools/tri/lean.t27
  lane improved specs/tools/tri/ledgers.t27
  lane improved specs/tools/tri/loop.t27
  lane improved specs/tools/tri/merging.t27
totals false inputs false verdicts false outputs false: IMPROVED_ONLY

Exit 3, IMPROVED_ONLY. The PR head 2e8539e is 967bb7b with master 408d740 merged in and the ledger rows recomposed; it changes no spec beyond master's. The two neutral rows are the two master commits between the base and the merge-base (#7436, #7866); their verdicts do not change.

base 76836e2 head 967bb7b
t27b pass / pass_vacuous (where the reference passes) 991 / 152 1002 / 141
reference pass 1179 1179
reference disagree (files / tests) 0 / 0 0 / 0
jit_interp_mismatch / crash 0 / 0 0 / 0

11 specs move from pass_vacuous to pass; none regresses.

Native arm64 CI: t27b-native-ratchet on 967bb7b reports reference disagree 0 files, 0 tests (of 1160 compared), and no finding on any of the 11 specs. It is red on master's own ledger findings, as on master.

Generated with Claude Code

@claude
specs(tools/tri): real tests for 11 tool cards, gen to merging (Closes …
0d2a995 
…#7903)

Each card held no test block and passed vacuously (0 runtime asserts) under
both the reference and t27b. The four tests check the rules the card is
written under, byte by byte:

- QUALIFIED_ID is REPO, ':' and ID (specs/tools/catalog.t27 ID_RULE);
- ID is tri/<name>, COMMAND is `tri <name>`, VARIANT is Commands::<Name>
  with <name> its kebab-case;
- the command name and every action are kebab-case, one ACTIONS_ABOUT per
  action, every ARGS entry a <positional> or a --flag;
- AGENTS holds the Queen (T) first, then the domain lead(s), no letter twice.

t27c test-report: 4/4 per spec, 0 vacuous. Sealed with seal --save and
checked with --verify. 13 hand mutants per card of the constants: every one
that applies is killed.

Closes #7903

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 8, 2026
Merged
Merged
Merged
Merged
Merged
Merged
@claude
t27b ledger: the 11 tool cards move from pass_vacuous to pass (Closes #…
967bb7b 
…7903)

docs/reports/t27b_expectations.json: `bless` of scripts/tri_loop/t27b.py
over master 76836e2's lab run, with this branch's t27b verdicts for the
11 specs (t27b test --check on the t27b lab: 4/4 each, 53 to 148 runtime
asserts), composed onto master's ledger with only these 11 rows:
pass_vacuous -> pass. pass 920 -> 931, pass_vacuous 151 -> 140,
max_not_pass unchanged at 36.

Closes #7903

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 8, 2026
Merged
Merged
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-08 19:28:39 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 38
PRs with All Checks Green 12
READY 1
FAILING 38
PENDING 0
NO CHECKS YET 0

These columns do not partition: 1 + 38 + 0 + 0 = 39, and there are 50 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=21a1173dfe32 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

This was referenced Oct 8, 2026
Open
Merged
Merged
Open
Merged
Merged
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-08 20:00:44 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 43
PRs with All Checks Green 7
READY 1
FAILING 43
PENDING 0
NO CHECKS YET 0

These columns do not partition: 1 + 43 + 0 + 0 = 44, and there are 50 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=3c0ade9e73e4 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

This was referenced Oct 8, 2026
Open
Merged
Open
Merged
Merged
Merged
Merged
Merged
Open
Open
Merged
Merged
@github-actions github-actions Bot mentioned this pull request Oct 8, 2026
Merged
@gHashTag
gHashTag enabled auto-merge (squash) October 8, 2026 21:43
@claude
Merge master into vac/tools-tri-1; recompose the 11 ledger rows (Closes
2e8539e 
#7903)

docs/reports/t27b_expectations.json conflicted with master's ledger
(408d740). Resolved from master's side: master's ledger with only this
branch's 11 rows recomposed by `bless` over this branch's lab run
967bb7b: pass 931 -> 942, pass_vacuous 151 -> 140.

Closes #7903

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot mentioned this pull request Oct 8, 2026
Merged
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-08 21:57:13 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 40
PRs with All Checks Green 10
READY 3
FAILING 40
PENDING 0
NO CHECKS YET 0

These columns do not partition: 3 + 40 + 0 + 0 = 43, and there are 50 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=3c0ade9e73e4 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

This was referenced Oct 8, 2026
Merged
Merged
Merged
@gHashTag
gHashTag merged commit 4daabae into master Oct 8, 2026
33 of 34 checks passed
@gHashTag gHashTag mentioned this pull request Oct 8, 2026
Merged
gHashTag pushed a commit that referenced this pull request Oct 8, 2026
@claude
Merge master 4daabae into t27b-ptr-eq-7742 (Closes #7742)
7a00602 
From master's side (#7904's tool-card tests and others; no t27b code changed, the conflict is the ledger only).
docs/reports/t27b_expectations.json: master's ledger plus this branch's three
rows, recomposed as before (gen_hazard_pointers.t27 -> pass, two new pass
rows): pass 942 -> 945, max_not_pass 33 -> 32. AGENTS.md: master's remainder plus this branch's clause,
re-measured with `wc -l` over master 4daabae: src/*.rs 15227 -> 15266,
src/*/*.rs unchanged, tests/*.rs 8164 -> 8172.

Merged-tree checks on the t27c lab: cargo test --release -p t27b 130
passed, 0 failed; under `t27b test --check` gen_hazard_pointers.t27 (4/4),
pointer_equality.t27, ptr_eq_plan.t27 and opaque_pointer.t27 pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gHashTag added a commit that referenced this pull request Oct 9, 2026
@gHashTag @claude
specs(tools/tri): real tests for 10 tool cards, misread to red (Closes …
6b6386e 
…#7951) (#7958)

Each card held no test block and passed vacuously (0 runtime asserts) under
both the reference and t27b. The same four tests as #7904 check the rules
the card is written under, byte by byte:

- QUALIFIED_ID is REPO, ':' and ID (specs/tools/catalog.t27 ID_RULE);
- ID is tri/<name>, COMMAND is `tri <name>`, VARIANT is Commands::<Name>
  with <name> its kebab-case;
- the command name and every action are kebab-case, one ACTIONS_ABOUT per
  action, every ARGS entry a <positional> or a --flag;
- AGENTS holds the Queen (T) first, then the domain lead(s), no letter twice.

t27c test-report: 4/4 per spec, 0 vacuous. t27b test --check: 4/4, 60 to
101 runtime asserts. Sealed with seal --save and checked with --verify.

docs/reports/t27b_expectations.json: `bless` of scripts/tri_loop/t27b.py
over lab run 967bb7b with this branch's t27b verdicts for the 10 specs,
composed onto master's ledger with only these rows: pass_vacuous -> pass.
pass 942 -> 952, pass_vacuous 140 -> 130.

Closes #7951

Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@gHashTag gHashTag mentioned this pull request Oct 9, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

t27b coverage: real tests for 11 tool cards in specs/tools/tri (gen to merging)

2 participants

@gHashTag @claude