Skip to content

feat(t27b): JIT on arm64 Linux, corpus --json/--runner, Railway lab - #6098

Merged
gHashTag merged 4 commits into
masterfrom
claude/t27b-railway-lab
Oct 4, 2026
Merged

feat(t27b): JIT on arm64 Linux, corpus --json/--runner, Railway lab#6098
gHashTag merged 4 commits into
masterfrom
claude/t27b-railway-lab

Conversation

@gHashTag

@gHashTag gHashTag commented Oct 4, 2026

Copy link
Copy Markdown
Owner

Closes #6071
Refs #6063

The owner's Mac runs at load 600-850, so t27b corpus runs move to a Railway service. The JIT is exercised on aarch64 Linux under qemu-user.

What changes

  • cli/t27b/src/jit.rs: the JIT works on arm64 Linux. The region is mapped RW, written, flushed (dc cvau / ic ivau using CTR_EL0 line sizes, dsb ish; isb), then switched to RX with mprotect. The macOS path (MAP_JIT, pthread_jit_write_protect_np, sys_icache_invalidate) is the same sequence, moved into map_code. The gate is target_arch = "aarch64" with target_os macos or linux. tests/differential.rs uses the same gate.
  • t27b corpus --json <path>: one record per file {file, reference, t27b, tests, invariants, blockers[], detail} plus totals and top_blockers. It is built from the structures the text summary already uses. corpus does not run the reference path, so reference is skip there and the lab fills it in.
  • t27b corpus --runner "<cmd> [args]": each per-file child is started through <cmd>. A container has no binfmt_misc, so this is how a qemu-user driver starts its aarch64 children.
  • contrib/railway/t27b-lab/:
    • a Dockerfile: rust:1-bookworm, the aarch64 cross gcc, qemu-user, and Zig 0.16.0 as in CI;
    • lab.py: builds t27c natively, cross-builds t27b, runs t27b corpus specs under qemu-user, runs t27c test-report <file> natively per file, then cargo test -p t27b under qemu-user. It serves /latest.json, /runs/<sha>.json, /runs/<sha>.log and /status.json. It re-runs when T27_REF moves (polled every 30 minutes).
    • No .sh files, no secrets. The clone is anonymous and the service never writes to GitHub.
  • docs/now/2026-10-04-t27b-railway-lab.md.

Lab

First full run, commit 9d87d14, 1190 files under specs/ (https://t27b-lab-production.up.railway.app/runs/9d87d14062f5dacf370a6a46da41596cbf89c40e.json):

count
reference pass / fail / blocked / timeout 646 / 25 / 516 / 3
t27b pass 47 (all 47 also pass the reference)
t27b mismatch / crash / timeout 0 / 0 / 0
cargo test -p t27b under qemu-user 18 passed, 0 failed

The first deploy's reference step failed with can't start new thread. The container reports 48 CPUs but has a 24-CPU quota and pids.max 1000, and each zig test spawns about one thread per visible CPU. Commit 9d87d14 sizes the workers from cpu.max and pids.max. A fork failure is counted as lab_error, never as pass or fail.

Note for #6064

#6064 (--reference / --reference-cache) is open and conflicts with master, so this PR does not stack on it. Whichever of the two merges second should fill the reference verdict into --json. lab.py already prefers the corpus JSON's reference when totals.reference.ran is true and runs t27c test-report itself only as a fallback.

Checks

  • cargo check -p t27b passes on macOS arm64 and for the aarch64- and x86_64-unknown-linux-gnu targets (--tests).
  • The aarch64 Linux run itself happened on Railway, as above.
  • tools/check_now_entry_shape.py: OK.

🤖 Generated with Claude Code

gHashTag and others added 3 commits October 4, 2026 21:01
@gHashTag @claude
feat(t27b): JIT on arm64 Linux, corpus --json/--runner, Railway lab
6b4b12c 
The t27b JIT now also runs on arm64 Linux: mmap RW, copy, clean the
data cache and invalidate the instruction cache to the point of
unification (dc cvau / ic ivau, CTR_EL0 line sizes, dsb ish; isb), then
mprotect RX. The macOS MAP_JIT path is the same sequence as before,
moved into map_code.

t27b corpus gains --json <path> (per-file records plus the totals the
text summary prints) and --runner "<cmd> [args]" (start each per-file
t27b test through a wrapper, for qemu-user without binfmt_misc).

contrib/railway/t27b-lab holds a Dockerfile and lab.py that build t27c
natively, cross-build t27b for aarch64-unknown-linux-gnu, run the
corpus under qemu-user and the reference path natively, and serve the
results over HTTP. No secrets; it never writes to GitHub.

Closes #6071
Refs #6063

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gHashTag @claude
fix(t27b-lab): size reference workers from the cgroup pids limit
9d87d14 
The first run on Railway failed the reference step with "can't start new
thread": the container reports 48 CPUs but has a 24-CPU quota and
pids.max = 1000, and each zig test spawns about one thread per visible CPU.
Workers now come from cpu.max and pids.max (overridable with T27_JOBS and
T27_REFERENCE_JOBS), both limits are recorded in the toolchain block, and a
fork failure is counted as reference lab_error, never as pass or fail.

Refs #6071

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gHashTag @claude
docs(now): first t27b lab numbers from Railway
f59fd76 
Refs #6071

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gHashTag gHashTag mentioned this pull request Oct 4, 2026
Open
6 tasks
@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-04 14:35:21 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 42
PRs with All Checks Green 8
READY 7
FAILING 42
PENDING 0
NO CHECKS YET 0

These columns do not partition: 7 + 42 + 0 + 0 = 49, and there are 50 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=a47525693295 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

This was referenced Oct 4, 2026
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
@gHashTag gHashTag mentioned this pull request Oct 4, 2026
Merged
@github-actions github-actions Bot mentioned this pull request Oct 4, 2026
Merged
@gHashTag @claude
Merge origin/master into claude/t27b-railway-lab (fold #6064 referenc…
5b287fa 
…e into corpus --json)

Resolves the cli/t27b conflicts with #6064 (corpus --blockers, --reference,
--reference-cache) keeping both feature sets. `corpus --json` now carries the
reference path's verdicts when `--reference` is given: per file `reference`
(pass / blocked / fail / timeout) plus `reference_detail`, and
`totals.reference` = {ran, pass, blocked, fail, timeout, skip}. lab.py reads
`reference_detail` when it takes the verdicts from the corpus JSON.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gHashTag

gHashTag commented Oct 4, 2026

Copy link
Copy Markdown
Owner Author

Merged master (a9650c0, #6064) in as 5b287fa, which is a merge commit with no rebase. I resolved the conflicts in cli/t27b/src/main.rs and README.md so both sets of options stay: --json/--runner and --blockers/--reference/--reference-cache. The reference result now goes into corpus --json. With --reference, each record has reference (pass/blocked/fail/timeout) plus reference_detail, and totals.reference = {ran, pass, blocked, fail, timeout, skip}. lab.py now reads reference_detail. I tested it on a one-file corpus with t27c, and cargo check plus cargo test -p t27b pass. Still to do: the lab does not pass --reference to its qemu corpus run yet, so it still runs its own native reference step. If it did, t27c and zig would be started from inside qemu-user, and nobody has tested that yet.

@github-actions github-actions Bot mentioned this pull request Oct 4, 2026
Merged
@gHashTag
gHashTag merged commit dd3864d into master Oct 4, 2026
28 of 30 checks passed
@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-04 15:57:01 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 42
PRs with All Checks Green 8
READY 6
FAILING 42
PENDING 0
NO CHECKS YET 0

These columns do not partition: 6 + 42 + 0 + 0 = 48, and there are 50 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=32e37152eb23 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

t27b lab: run the corpus on Railway (aarch64-linux under qemu-user)

1 participant

@gHashTag