fix(ci): judge the headless test graph instead of piping it through tee (Refs #989)

[dmitrii-f-t27]

S12 of #988, first part (Refs #989; Refs #616 -- this supersedes the tee fix of #987 by judging the result instead of only propagating it).

What was wrong

ci.yml ran the tests as zig build test -Dci=true 2>&1 | tee test-output.txt under bash without pipefail, so the step's exit code was tee's. On main (afc9d38) the test graph does not compile five of its roots, and every CI run reported that as a success. The summary line says it plainly and nothing read it:

Build Summary: 124/135 steps succeeded; 5 failed; 2510/2510 tests passed

Measured twice with Zig 0.15.2: the Build & Test job of CI run 36875247882 (Linux x86_64) and a Linux aarch64 container with empty caches. The five roots, and what the compiler says:

test root	error	capability card (S01 manifest)
src/trinity.zig -- the library's own test root	src/hybrid.zig:4, src/vsa.zig:7: no module named zig-hdc-vsa (the test module provides it as hdc_vsa); src/science.zig:53,143,161,179,204: expected optional type, found [59049]i8	trinity/lib.trinity
src/vm.zig	src/vm.zig:1150: expected optional type, found [59049]i8; src/vsa_jit.zig:117: member function expected 1 argument, found 2	trinity/lib.trinity
src/queen_api.zig	the C allocator without libc (4 errors): its test module does not set link_libc, as the main test module does	trinity/queen.lib
src/c_api.zig	src/c_api.zig:270: expected statement, found a document comment	trinity/abi.c-api
src/e2e_test.zig	imports packed_trit.zig, removed in 42490a2 (#517)	trinity/examples.zig

One more run step compiles and runs no test at all (external/zig-hslm/src/root.zig has no test declaration).

What this adds

• specs/reproduce/headless.t27 -- the headless profile as measured: Zig 0.15.2, the two commands, the four dependency pins of build.zig.zon, the 46 files the build installs (45 executables and libtrinity.a), the 67 test roots the test step compiles, the floors the graph may not drop below (135 steps, 61 run steps with tests, 2510 tests passed, at most 1 empty run step), and the five blocked roots, each with its error and the card it holds back.
• tools/reproduce.py -- the documented clean-clone command, python3 tools/reproduce.py headless. It points ZIG_GLOBAL_CACHE_DIR and ZIG_LOCAL_CACHE_DIR at a new empty directory, runs the build and the test step, reads each exit code from the process, and judges: exactly the blocked roots may fail (a sixth fails the gate, and so does a blocked root that compiles again -- take it off the list); a failed test, a leak, a failure the output does not name, a missing install, a changed dependency pin, fewer tests, fewer run steps or a declared root that left the graph fail it. Every install is hashed into the record it writes. --self-check plants fifteen defects (the swallowed exit code of CI: main has no build.zig matching its own source tree (2217 files vs a build script from March) #616 among them) and shows each caught. Standard library only.
• ci.yml: the Test step is now python3 tools/reproduce.py test --reuse-cache -- the same judge over the same test graph, the leak check kept. CI stays green on main only because the five failures are named, with reasons, in a file a reviewer can read; a new one turns it red.
• .github/workflows/reproduce.yml: the headless profile from a clean clone on a runner with no cache (setup-zig use-cache: false, empty zig caches), the negative control first, the record uploaded as reproduce-headless. On pushes to main, on pull requests that touch the build, weekly, and by hand.

Python follows the existing CI scripts (#997); .claude/rules/no-shell-scripts.md still says "zero Python", which the repository no longer follows.

What it does not do yet (rest of #989)

• Fix the five roots. Each is a small, separate change (a link_libc, a comment, a removed import, a module name, an optional that became an array); each makes one capability's tests run for the first time in months, and the gate will ask for its line to be removed.
• Consume the canonical t27 manifest by a pinned, byte-checked lock, and publish the capability -> spec -> artifact -> evidence index; the end-to-end task over CLI/MCP, tri-api policy, persistence and the Queen lifecycle; the catalog refresh and publication.
• Any profile but headless.

{
  "version": 1,
  "head_sha": "14c595aae4b13fb5f60d729578d46046c2fce8d3",
  "summary": "CI judges the headless test graph against a declared profile instead of piping it through tee, so the five test roots that do not compile on main are named, with reasons, and a sixth fails the job.",
  "changes": [
    "Added specs/reproduce/headless.t27: the headless profile measured at afc9d384 with Zig 0.15.2, its commands, dependency pins, 46 installs, 67 compiled test roots, graph floors and five blocked roots with errors and capability cards.",
    "Added tools/reproduce.py: the clean-clone command that builds and tests with fresh zig caches, reads every exit code from the process and judges the result against the spec, with a self-check of fifteen planted defects.",
    "Replaced the tee-piped CI test step with the same judge, keeping the leak check, and added a reproduce workflow that runs the whole profile on a runner with no cache and uploads the record."
  ],
  "tests": [
    {
      "command": "python3 tools/reproduce.py --self-check",
      "status": "passed",
      "result": "All fifteen planted defects are reported under their own codes, including the swallowed exit code of #616, and a conforming record stays silent.",
      "evidence": "tools/reproduce.py self_check; run locally on macOS Python 3 before the commit"
    },
    {
      "command": "python3 tools/reproduce.py headless --zig zig-0.15.2 in ubuntu:24.04 aarch64 with empty caches",
      "status": "passed",
      "result": "Build 94/94 steps with 46/46 installs; test 124/135 steps with exactly the five blocked roots failing, 61 run steps with tests, one empty, 2510 tests passed; no violation.",
      "evidence": "Run on tree e5bb811d4eaf (this commit) in ubuntu:24.04 aarch64 with empty caches, 2 min 43 s; record zig-out/reproduce/headless.json lists no violation"
    },
    {
      "command": "Build and Test job of CI run 36875247882 on main afc9d384, read from its log",
      "status": "passed",
      "result": "The log prints the same summary, 124/135 steps succeeded with five failed compiles, while the job reported success through tee.",
      "evidence": "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/gHashTag/trinity/actions/runs/36875247882"
    }
  ],
  "limitations": [
    "The five blocked roots are named, not fixed; each fix is a separate small change that makes one capability's tests run again and removes its line from the spec.",
    "Only the headless profile is reproduced; web, native, FPGA, training and network profiles keep their present status.",
    "The pinned t27 lock, the capability evidence index, the end-to-end task and the catalog publication of #989 come in later parts."
  ],
  "tags": [
    "Engineering",
    "Verification",
    "CI"
  ],
  "blog": {
    "title": "A green build that hid five test roots that never compiled",
    "summary": "Trinity's test step piped its output through tee without pipefail, so failed compiles of five test roots, the core library among them, reported success on every push until a judge read the exit code.",
    "outline": [
      "How a pipe without pipefail turns a failed test step green, and what the build summary said all along.",
      "Declaring the headless profile as data: commands, pins, installs, test roots, floors and the blocked roots with their reasons.",
      "What is still open: fixing the five roots one by one, the pinned contract lock, the evidence index and the end-to-end task."
    ]
  }
}

🤖 Generated with Claude Code