Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 18 additions & 2 deletions trios/agent-server/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -192,9 +192,25 @@ WORKDIR /app
#
# tini as PID 1 reaps orphans and forwards signals. It is what `docker run
# --init` installs, and it is the standard answer to exactly this.
#
# python3 is here for the same reason perl-base is: declared, not assumed.
# Every gate that decides whether a bee's pull request lands is a Python
# script -- tools/dupe_scan.py is the Duplicate Body Ratchet,
# tools/check_assertionless_spec_tests.py is the corpus ratchet,
# tools/ci/check_specs_still_parse.py is the required Spec Parse Ratchet, and
# the twenty-one reporting helpers under scripts/tri_loop/ are Python too. A
# bee that cannot run the gate that will fail it is a bee guessing, and the
# brief has been telling it to run `python3 tools/dupe_scan.py --name <fn>`
# since 2026-09-20.
#
# `python3`, not `python3-minimal`: those scripts import json, hashlib,
# difflib and subprocess, and Debian's minimal package carries a subset of the
# standard library chosen for dpkg's maintainer scripts. Nothing here needs a
# module from outside the standard library, so there is no pip and no venv.
RUN apt-get update \
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client tini perl-base \
&& rm -rf /var/lib/apt/lists/*
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client tini perl-base python3 \
&& rm -rf /var/lib/apt/lists/* \
&& python3 --version

# An unprivileged account for the agents' shell commands.
#
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -152,26 +152,86 @@ export const CRITERION_PROGRAMS: ReadonlySet<string> = new Set([
/**
* The `t27c` subcommands a criterion may name.
*
* `t27c --help` lists 180 of them, and they are not all readers. `battery` and
* `t27c --help` lists 155 of them, and they are not all readers. `battery` and
* `gates` RUN every `check_*.py` in the tree, `silicon` drives a bitstream
* build, `fpga-flash` writes to hardware, `serve` and `bridge` open sockets,
* `fmt`, `rename` and `seal` edit files. The tree they would act on is the bee's
* own commit, so an allowlist of programs alone hands a bee the ability to run
* a script it just wrote by putting `t27c battery` in an issue's criteria.
*
* So the subcommand is allowlisted too, to the ones that only READ a spec and
* print an answer - which is all an acceptance criterion in this repository has
* ever needed (measured over the open backlog: spec-status, gen, parse and
* typecheck cover every criterion that names the compiler).
* So the subcommand is allowlisted too. What belongs on this list is a command
* that READS a spec, prints an answer, and finishes well inside
* CRITERION_COMMAND_TIMEOUT_MS. The first version held five names, and the
* criteria written against it were greps - which is how 388 of the 837 specs
* the oracle reaches came to satisfy every criterion ever filed against them
* and still generate Zig that does not compile.
*
* Measured 2026-09-20 against master, on the corpus, per command: everything
* below except `test-report` answers in under 30 s and most in under one.
*
* `test-report` is the one that runs anything: it generates the reviewed spec
* and asks Zig to build and run that spec's own tests, which is the question
* the oracle asks and the only one that distinguishes a file that greps clean
* from a file that works. It is admitted deliberately. The bee already has a
* shell in this container - that is what a bee IS - so nothing here is reachable
* to it that was not reachable before; what the allowlist prevents is a
* criterion RIGGING its own measurement by running a script from the commit
* under review, and `test-report` runs the compiler on one named spec.
*
* Deliberately absent, with the reason:
* - `dupes` (~50 s) and `check-calls` (~140 s) read the whole corpus. Past
* the timeout a criterion is inconclusive, which spends a review and
* establishes nothing. A bee may still run them; its shell has no
* allowlist.
* - `corpus`, `backlog`, `suite`, `battery`, `gates`: minutes, or they
* execute the tree.
* - `fmt`, `rename`, `seal`, `init`, `compile`, `doc`: they write.
*/
export const CRITERION_T27C_SUBCOMMANDS: ReadonlySet<string> = new Set([
// The verdicts.
'spec-status',
'impl-status',
'classify',
'parse',
'parse-complete',
'parse-conform',
'typecheck',
'test-report',
// What the spec declares.
'symbols',
'inspect',
'outline',
'exports',
'depends',
'test',
'tree',
'strings',
// What it measures.
'count',
'loc',
'size',
'metrics',
'depth',
'stack',
'hash',
'coverage',
// What is wrong with it.
'lint',
'deadcode',
'orphans',
'spellcheck',
'validate',
'validate-vacuity',
'todo',
'check-deps',
'deps-tree',
'analyze',
// Comparisons and evaluation.
'xref',
'api-diff',
'diff',
'eval',
// What the generators emit.
'gen',
'gen-c',
'gen-rust',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -260,7 +260,7 @@ describe('criteria become checks', () => {
})

describe('t27c is allowed only where it reads', () => {
// `t27c --help` lists 180 subcommands and they are not all readers: `battery`
// `t27c --help` lists 155 subcommands and they are not all readers: `battery`
// and `gates` run every `check_*.py` in the tree, and the tree is the bee's
// own commit. A program allowlist alone would let a criterion run a script
// the bee wrote in the same attempt.
Expand All @@ -275,6 +275,13 @@ describe('t27c is allowed only where it reads', () => {
['t27c fpga-flash'],
['t27c'],
['grep -c x specs/a.t27 && t27c battery'],
// Readers, and still refused: both walk the whole corpus. Measured
// 2026-09-20 on master, `dupes` takes ~50 s and `check-calls` ~140 s
// against a 60 s per-command timeout, and a criterion that times out is
// inconclusive - it spends a review and establishes nothing. The bee's own
// shell has no allowlist and can run either.
['t27c dupes --name magadd'],
['t27c check-calls --specs-dir specs'],
])('refuses %s', (cmd) => {
expect(isSafeCommand(cmd)).toBe(false)
})
Expand All @@ -285,6 +292,18 @@ describe('t27c is allowed only where it reads', () => {
['t27c parse specs/a.t27 2>&1 | grep -c x'],
['t27c typecheck specs/a.t27'],
['t27c impl-status'],
// The compile question, which no grep can ask: `test-report` generates the
// reviewed spec and has Zig build and run that spec's own tests. Admitted
// deliberately - see the comment on CRITERION_T27C_SUBCOMMANDS.
['t27c test-report specs/a.t27 2>&1 | grep -c BLOCKED'],
// The readers a criterion could not name until now.
['t27c coverage specs/a.t27'],
['t27c lint specs/a.t27 | grep -c WARN'],
['t27c symbols specs/a.t27 | wc -l'],
['t27c outline specs/a.t27'],
['t27c validate-vacuity --specs-dir specs'],
['t27c api-diff specs/a.t27 specs/b.t27'],
['t27c eval 1 + 2'],
])('allows %s', (cmd) => {
expect(isSafeCommand(cmd)).toBe(true)
})
Expand Down
Loading