feat(reproduce): index every capability card against a run of the profile (Refs #989)

[dmitrii-f-t27]

S12 of #988, fourth part (Refs #989). It covers deliverable 3, a machine-readable capability -> spec -> generated artifact/adapter -> build/test -> evidence index, and acceptance criterion 4, every capability measured complete, explicitly blocked or intentionally excluded with reason; no coverage percentage uses catalog counts.

What this adds

• tools/capabilities.py index --record R holds every S01 capability card (external/t27, locked by feat(reproduce): consume the t27 contracts byte for byte at a pinned revision (Refs #989) #1213) to this tree and to a record of the headless profile. In order, it:

  1. verifies the vendored contracts against the lock, and refuses to index unverified cards;
  2. imports the S01 checker itself (external/t27/tools/trinity_manifest.py) for its inventory and check of this tree, so target ownership, paths and dialects are judged by the canonical code, not a copy;
  3. reads the record (installs with sha256, compiled test roots, the test verdict) and refuses a record of another revision;
  4. runs the planned acceptance commands.

  It writes capabilities-<arch>.json and a table for the job summary.

• specs/reproduce/capabilities.t27 is only the measuring, never a restatement of the cards:

  • which acceptance commands run (RUN, with -Dci=true added, stdin closed, bounded);
  • which do not, and why (NOT_RUN);
  • the capabilities known to be blocked, each with what blocks it (KNOWN_BLOCKED);
  • the S01 checker findings that are drift since its pinned inventory (DRIFT_CODES).

  Every headless card with an acceptance command sits in exactly one place: the profile's own commands, RUN or NOT_RUN.

• .github/workflows/reproduce.yml: both legs (x86_64, aarch64) index the capabilities after the headless run.

How a card is decided

status	when
excluded	its disposition is deprecated, out-of-scope, catalog-only or external (the card's own reason); its profile is not headless; or nothing the profile builds, tests or runs measures it
measured	every exe/lib it owns that the profile installs is in the record with its sha256; every root of the profile's test step it owns compiled with no failure; its acceptance is the profile's command, or ran and exited 0, or is declined with a reason. At least one of these must be evidence: an install hashed, a test root passed, or a command that exited 0
blocked	any of that failed. Each blocked card must be in KNOWN_BLOCKED, and each card there must still be blocked

The checker's findings fail the index, except PIN_MISMATCH and COUNT_MISMATCH: S01 pinned an older revision of this tree, and those are reported as drift.

At this head

Linux aarch64, head 48dfcc5dea5f, a fresh headless run: 51 cards: 21 measured, 3 blocked, 27 excluded with a reason. CI writes the same table for both architectures into each job summary.

	capabilities
measured (21)	abi.c-api, agent.daemons, agent.phi-loop, agent.tri-api, api.http-server, bench.suite, bot.tri-bot, cli.tri, convert.b2t, examples.zig, lib.trinity, llm.firebird, llm.igla-chat, node.trinity-node, ops.railway-cli, queen.lib, research.programs, search.vsa-index, test.graph, train.hslm, vibee.compiler
blocked (3)	mcp.needle-mcp, mcp.trinity-mcp, registry.commands (each with its reason below)
excluded: disposition catalog-only (1)	catalog.spec-mirror
excluded: disposition deprecated (3)	archive.legacy, pkg.vendored-cache, specs.t27-vendored-compiler
excluded: disposition external (2)	numeric.golden-float, vsa.zig-hdc
excluded: nothing measures it (8)	release.packaging, research.unreferenced-sources, specs.tri-corpus, specs.vibee-corpus, state.trinity-dir, tools.scripts, tri27.programs, tri27.toolchain
excluded: profile fpga (2)	fpga.adapter, fpga.in-repo
excluded: profile native (4)	ext.vscode-swe, native.canvas, native.node-gui, native.queen-app
excluded: profile network (1)	ops.deploy-infra
excluded: profile research (1)	research.nexus-docs
excluded: profile training (1)	training.targets
excluded: profile web (4)	catalog.world-scan, web.docsite, web.queen-web, web.site

What the measuring found:

• mcp.needle-mcp and mcp.trinity-mcp cannot pass their acceptance. zig build needle-mcp and zig build trinity-mcp name steps that build.zig comments out (around lines 1507 and 1551). The S01 inventory still counts both steps, because tools/trinity_manifest.py does not skip // lines, and both cards claim measured.
• registry.commands cannot pass its acceptance as written. zig build export-registry && git diff --exit-code .trinity/registry.json fails every time, because the export writes the time it ran into generated_at.
• The S01 cards' measured meant that one CI run built the tree. The index replaces it per capability, with installs hashed, test roots passed and commands run on this tree. For example, abi.c-api and queen.lib, both declared, now measure: their libraries build and the C API tests pass (fix(build): compile the five blocked test roots again (Refs #989) #1210).
• release.packaging owns only step:release and names no command, so nothing measures it.
• The inventory's count of canonical .t27 files includes the 70 vendored contract copies, because S01 excludes only the website mirror. That is reported as drift, and it is something to fix in the S01 checker.

Negative controls

• --self-check runs the decision on toy cards and plants every rule once: a missing install, a failed or uncompiled test root, a failing or missing run, a known block that now measures, an unplanned acceptance, a card both run and not run, a plan line for no card, a failed record, a record of another revision, and a checker finding that is not drift. It fails if any rule never fired.

Not yet (rest of #989)

• The end-to-end task (CLI/MCP, tri-api tool policy, state persistence, Queen lifecycle) with independent expected outputs, and its negative mutations of a permission decision and an output.
• The catalog refresh and publication.
• The two t27-side findings above (the checker counts commented steps and vendored copies). They are recorded here, not fixed.

{
  "version": 1,
  "head_sha": "b1cb56585d24c83616063fabbf7f26d629ffef43",
  "summary": "Every S01 capability card is now held to this tree and to a run of the headless profile, and comes out measured, blocked with its reason, or excluded with its reason, in a JSON index and a job-summary table.",
  "changes": [
    "Added tools/capabilities.py: verifies the contract lock, imports the vendored S01 checker for the inventory and check of this tree, reads a headless record, runs the planned acceptance commands, and decides each card.",
    "Added specs/reproduce/capabilities.t27: the commands run, the ones not run and why, the known blocks with their reasons, and the checker findings that are drift.",
    "Made both legs of reproduce.yml build the index after the headless run and upload capabilities-<arch>.json with the record."
  ],
  "tests": [
    {
      "command": "python3 tools/capabilities.py --self-check",
      "status": "passed",
      "result": "Every rule of the decision planted on toy cards fired: a missing install, a failed or uncompiled test root, a failing or missing run, a known block that measures, an unplanned or conflicting plan line, a line for no card, a failed record, a record of another revision, a checker finding that is not drift.",
      "evidence": "Run locally on macOS Python 3 before the commit"
    },
    {
      "command": "python3 tools/reproduce.py headless, then python3 tools/capabilities.py index --record R, twice, in ubuntu:24.04 aarch64",
      "status": "passed",
      "result": "51 cards: 21 measured, 3 blocked (all in KNOWN_BLOCKED), 27 excluded with a reason; no violation; the second index run also exits 0 and leaves no file in external/t27.",
      "evidence": "Linux aarch64 container at 48dfcc5d, the same tree (a1d26fc6) as b1cb56585d24, whose commit only changes the message"
    }
  ],
  "limitations": [
    "The two S01-side defects it found (commented steps counted, vendored copies counted as canonical .t27) are recorded, not fixed in t27.",
    "Acceptance commands that start services or need credentials are declined with a reason, and those capabilities are measured by their builds and installs only.",
    "The end-to-end task and the catalog publication of #989 come next."
  ],
  "tags": [
    "Engineering",
    "Verification",
    "CI"
  ],
  "blog": {
    "title": "51 capability cards, each measured, blocked or excluded",
    "summary": "The S01 cards said measured when one CI run built the tree; the index now holds each card to the installs, tests and commands of a clean run, and names what it cannot measure.",
    "outline": [
      "What measured used to mean in the S01 cards -- one CI run built the tree -- and what it means for each capability now: installs hashed, test roots passed, commands run.",
      "The three blocked capabilities: two MCP steps that build.zig comments out while the inventory still counts them, and a registry check that writes its own time.",
      "Twenty-seven exclusions, each with its reason -- a disposition, a profile S12 does not reproduce, or nothing that measures it -- and no figure taken from the catalog."
    ]
  }
}

🤖 Generated with Claude Code