Refs #6434
Refs #6063
Problem
"mismatch 0" on the t27b lab is not a comparison with the reference path.
- In
cli/t27b/src/main.rs (the --check block), MISMATCH means the JIT disagrees with eval.rs::Interp. Both read the same IR from lower.rs. If lower.rs mistranslates t27 semantics (wrap vs trap, promotion, float rounding, layout), both sides compute the same wrong thing and agree.
- The only comparison with the reference is per file.
blockers.rs::parse_test_report keeps only the tests and FAIL totals of t27c test-report, even though t27c already prints each test's outcome (bootstrap/src/test_report.rs, Report.outcomes).
- steward.t27
is_alarm(fail, fail) is false, so the 7 files where both fail (lab run 03362c1) count as agreement without checking that the same test failed. Example: specs/port/trinity/src/tri/gen_vector2d.t27: t27b fails Vec2_length with assert_eq left 4627730092099942521 vs right 4627730092099895296 (one f64 ulp). The reference says only "1 of 4 tests fail".
Stop::Fuel and Stop::Depth are also counted as agreement whatever the JIT did.
Change
parse_test_report returns the per-test outcome list. The reference cache stores it (new row format; old rows are read as file-level only).
t27b corpus --reference compares test by test. A test whose t27b verdict differs from the reference verdict is ref_mismatch, a new verdict counted in summary.ref_mismatch and treated by the doctor as LAB-MISMATCH.
- steward.t27
is_alarm judges per test when per-test data exists.
- Interpreter Fuel/Depth stops report
unknown, not agreement.
Done when
- The lab JSON carries
ref_mismatch per file and in the summary, and the 7 fail/fail files show which tests disagree, if any.
- steward.t27 has tests for the per-test
is_alarm.
- New subcommand
tri t27b diff <spec> prints both verdict lists side by side.
🤖 Generated with Claude Code
Refs #6434
Refs #6063
Problem
"mismatch 0" on the t27b lab is not a comparison with the reference path.
cli/t27b/src/main.rs(the--checkblock), MISMATCH means the JIT disagrees witheval.rs::Interp. Both read the same IR fromlower.rs. Iflower.rsmistranslates t27 semantics (wrap vs trap, promotion, float rounding, layout), both sides compute the same wrong thing and agree.blockers.rs::parse_test_reportkeeps only thetestsandFAILtotals oft27c test-report, even though t27c already prints each test's outcome (bootstrap/src/test_report.rs,Report.outcomes).is_alarm(fail, fail)is false, so the 7 files where both fail (lab run 03362c1) count as agreement without checking that the same test failed. Example:specs/port/trinity/src/tri/gen_vector2d.t27: t27b failsVec2_lengthwith assert_eq left 4627730092099942521 vs right 4627730092099895296 (one f64 ulp). The reference says only "1 of 4 tests fail".Stop::FuelandStop::Depthare also counted as agreement whatever the JIT did.Change
parse_test_reportreturns the per-test outcome list. The reference cache stores it (new row format; old rows are read as file-level only).t27b corpus --referencecompares test by test. A test whose t27b verdict differs from the reference verdict isref_mismatch, a new verdict counted insummary.ref_mismatchand treated by the doctor asLAB-MISMATCH.is_alarmjudges per test when per-test data exists.unknown, not agreement.Done when
ref_mismatchper file and in the summary, and the 7 fail/fail files show which tests disagree, if any.is_alarm.tri t27b diff <spec>prints both verdict lists side by side.🤖 Generated with Claude Code