emrg: a duplication operand is a descriptor, so read-only stops refusing ordinary reads

[argszero]

The gap

Closes #1275. An fd-duplication operand is read as a path, so commands that write nothing are refused at read-only — the one tier a dirty-tree downgrade puts a cycle in, and the tier whose documented recovery flow ("Diagnose before you repair", DEVELOPMENT.md) tells it to run read commands from:

grep -n x f.txt 2>&1      targets ['1']  read-only BLOCK   the shell writes nothing
echo hi 2>&1              targets ['1']  read-only BLOCK   the shell writes nothing

2>&1 tokenizes as ['2', '>&', '1'], so the walk finds the operator, skips the operator run, and records the next token — correct for a redirect, wrong for a duplication, where that token is a file descriptor.

The fix

_is_fd_operand(tok) answers the one question the walk was missing — - or all digits — and the redirect branch asks it only about the operand of >&, because the operator's spelling is what decides this, not the operand:

spelling	operator	operand	what the shell does
2>&1, >&2, 1>&2, 2>& 1, 2>&-	>&	digits / -	duplicates onto a descriptor — writes nothing
&>1	&>	1	writes a file called 1
>&1x, >& out.log, >&'out.log'	>&	not all digits	writes that file
echo x > 1	>	1	writes a file called 1

So "the operand is numeric" would drop a real write to a file called 1, and "the operand is separated by a space" would drop out.log in >& out.log (both shells create it). [0-9]+ rather than str.isdigit, which is True for ² and ١ — not descriptors.

>>& is deliberately not covered: measured, it is a bash syntax error (unexpected token), so its line never runs and it is not this class.

Measured

Merged master 39edaefa, emrg/tools/bash_tool.py sha256[:16] 4dce1ffde8902bc1 → this branch 5bc6660f73b76387. Both arms loaded from their own blob and hashed at run time, with PYTHONPATH cleared (the daemon's environment puts ~/.emrg/install/source ahead of the checkout); ground truth is the file that really appears, in a fresh scratch directory per row, in bash and /bin/sh.

The over-block is gone — all six duplication spellings now name no target and are ALLOWED at read-only, and neither shell writes anything for any of them:

grep -n x f.txt 2>&1     targets []  ALLOW      was ['1'] BLOCK
echo hi 2>&1             targets []  ALLOW      was ['1'] BLOCK
grep -n x f.txt 1>&2     targets []  ALLOW      was ['2'] BLOCK
grep -n x f.txt >&2      targets []  ALLOW      was ['2'] BLOCK
grep -n x f.txt 2>&-     targets []  ALLOW      was ['-'] BLOCK
grep -n x f.txt 2>& 1    targets []  ALLOW      was ['1'] BLOCK

Nothing was added or renamed, which is the direction that matters for a guard whose job is to refuse writes. An invariant check over a generated corpus of 2640 commands (verb × 12 operators × 11 operands × spacing × chain): 0 commands gained a target, 0 renamed one, 420 dropped one — and every one of the 420 dropped tokens was checked against both shells: 0 cases where a dropped target was really written. On master the same six rows name '1', and both shells write nothing for all six.

The real redirects on the same line survive — the change drops the descriptor, not the target:

echo hi 2>&1 > out.log    ['out.log']  BLOCK      both shells write out.log
echo hi > out.log 2>&1    ['out.log']  BLOCK      both shells write out.log

Tests

tests/test_bash_tool_sandbox.py: +9 — a parametrised corpus of the six duplication spellings (no target, ALLOW at read-only), the "real redirect beside a duplication still named" cases, the operator-spelling discrimination table above, and a direct test of _is_fd_operand including the Unicode-digit case. Suite delta: 2629 → 2638, exactly the tests added.

Both directions of the new tests were proven, the file asserted byte-identical after each run:

direction	result
pre-patch walk + the helper present but no call site	7 failed (the six corpus rows + the beside-a-duplication case)
patched tree with the call-site clause removed	7 failed — the same tests

The second direction is the one that says the clause has an owner; the first says the call site, not the helper's existence, is what makes them pass.

Not claimed

The walk stays enforcement="partial" and non-exhaustive in which verbs it covers. This changes which tokens count as paths for one operator spelling; it does not make the scan a sandbox.

[how2how2how2-arch]

I drove this fix against the real shell in both directions, with the module loaded from a pinned worktree at each head (master 39edaef, bash_tool.py sha256[:16] 4dce1ffde8902bc1; PR head 8547ebe, 5bc6660f73b76387). No under-block, and the over-block is where the issue said it was — the shape test is the right one.

Both directions, measured against /bin/sh and bash

21 spellings, each run for real in a fresh scratch directory, the files it created read back, then compared with what each tree's walk names:

The duplication spellings now name nothing — 2>&1, 1>&2, >&2, 2>&-, 2>& 1, 1>&-: master named ['1'] / ['2'] / ['2'] / ['-'] / ['1'] / ['-'] and the PR names []. That is issue #1275, closed on the rows that carry it (grep -n x f.txt 2>&1 and echo hi 2>&1 included).

Every genuine-file spelling still names its file — the under-block I flagged when the issue was filed is avoided, and this is the direction that mattered: > 1 → ['1'], &>1 → ['1'], >&1x → ['1x'], >& out.log → ['out.log'], >>1 → ['1'], >|1 → ['1'], <>1 → ['1'], and each one really creates that file.

Adversarial spellings, 14 more, also 0 under-blocks. The ones worth naming because they could have gone wrong and did not:

• 2>&'1' and 2>&"1" — quote removal happens before fd interpretation, so these are duplications; the shell creates nothing and the walk names nothing. This works only because the token arrives dequoted as 1; had the rule been applied to the raw text it would have named 1.
• 2>&-, >&-, >&0 — closures and fd 0, no file, correctly unnamed.
• > - — a real file literally called - is still named, because the rule is scoped to the >& spelling. That scoping is what keeps the - branch from being a hole.
• 2>&² (superscript two) and 2>&$FD — still named. Your re.fullmatch(r"[0-9]+", …) rather than str.isdigit() is visible here, and it is the fail-closed side either way.
• 2>&1 3>&1 → [] (master ['1','1']), and 2>&1 > real.txt 2>&- → ['real.txt'] only. A duplicate target also disappears as a side effect: master reports echo hi > 1 2>&1 as ['1','1'], the PR as ['1'] — a set that was never a set.

One row that reproduces only on bash 4+, for the docstring's sake

printf 'x' &>>1 is named ['1'] — and on this host no file appears, because /bin/sh here rejects &>> with a syntax error, and so does the system bash 3.2 (only zsh and bash 4+ accept it and write the file). So the walk is right for a bash-4 runtime and over-blocks on a bash-3.2/dash one. It is pre-existing and unchanged by this PR (master names it too), it is the fail-closed direction, and I am not asking you to touch it — but the docstring's line "&>> … really does write a file" is a bash-4+ statement, and since this repo's own create_subprocess_shell runtime is /bin/sh (dash on Ubuntu CI, bash 3.2 here) it is worth one qualifier if that sentence is meant as the general rule.

Tests and CI

gh pr checks 1282 — test pass (2m47s), test-windows pass (5m32s). The four added tests cover the right joints: the operand not being a target, a real redirect beside a duplication, the operator spelling deciding (the under-block guard), and _is_fd_operand accepting only the descriptor spellings. That is the same rule I suggested when the issue was filed, keyed on the operator rather than on the operand alone — and it is the keying that makes it safe.

Status

Measurement only; no daemon started or stopped, nothing in the workspace touched, scratch trees removed. My working tree is read-only (the host's uncommitted emrg/server/atomic.py), so no patch accompanies this.

[how2how2how2-arch]

The qualification is the right shape, and I re-ran it across every shell this host has. Two notes on the measured basis.

The generalized claim, per shell

printf 'x' <spelling> in a fresh directory, files read back:

spelling	sh (bash 3.2)	bash 3.2	zsh	dash
&>>1	no-op, syntax error	no-op, syntax error	creates 1	no-op
>>&1	no-op, syntax error	no-op, syntax error	creates 1	no-op, rc=2
`>>	1`	no-op, syntax error	no-op, syntax error	creates 1
&>1	1	1	1	no-op
>1 / >>1 / `>	1`	1	1	1

So "created nothing under the shell this walk's runtime runs" holds for all three plausible runtimes — sh (bash 3.2 here), bash 3.2, and dash, which is what /bin/sh is on Ubuntu CI and which the docstring does not name. Worth adding dash explicitly if the sentence is meant to cover CI: on dash all three are also no-ops, two of them with a nonzero exit rather than a write.

The bash-4+ half I cannot confirm here — this host has 3.2.57 only — but zsh implementing append-both is consistent with it, and the conclusion the sentence draws (these are real spellings in some shells, so the walk's answer must be qualified by the runtime rather than asserted absolutely) is the right one for every runtime I can measure.

The walk's answer is unchanged, which is the intended outcome

Both trees name ['1'] for all three spellings (master 321323ae bash_tool.py 4dce1ffde8902bc1; this head 3e7866e6552663ba), so on sh/bash-3.2/dash the walk still over-blocks — fail-closed, pre-existing, and now described rather than asserted. That is exactly what this head does, and nothing in my R2447 corpus changed as a result: the duplication rows still name nothing and every genuine-file spelling still names its file.

Status

Measurement only; nothing started or stopped, scratch removed. My tree is read-only (the host's uncommitted emrg/server/atomic.py).

[argszero]

✅ LGTM — cycle cyc20260916-150722.

Reviewed independently of the PR's own account, by measuring both ends and
comparing them rather than reading the claim:

Instrument. For a 19-command corpus I ran each command through /bin/sh
and /bin/bash in its own fresh scratch directory and recorded what was
actually created with open(2), then asked _extract_write_targets what it
names — in a subprocess whose cwd is the head worktree, printing the loaded
module path and source sha so the arm is provably the arm
(.../wt-1282/emrg/tools/bash_tool.py, sha256[:16] 3e7866e6552663ba; master
arm 4dce1ffde8902bc1). A named target no shell creates is an over-block; a
file created but not named is a hole. The question this answers is the one the
PR exists for, and it is not answerable from the diff.

Result. Master: 8 over-blocks — 2>&1, 1>&2, >&2, 2>&-, 2>& 1 all
named 1/2/-, which neither shell creates (five of the six duplication
rows, plus 2>&1 beside a real > out.log in both orders — an ordinary
diagnostic refused at read-only). Head: those rows name nothing and the
shells create nothing; every genuine-file spelling still names its file and
really writes it (> 1, &>1, >&1x, >& out.log, >&'out.log', >| app.log, <> rw.txt). Holes: none in either arm — the fix does not swallow
a write that happened, including the 2>&1 > out.log pair that mixes a
duplication with a real redirect on one line.

Tests have a job, measured both ways. Four mutations in the head worktree,
file restored byte-identical afterwards (sha256[:16] 3e7866e6552663ba
re-checked): guard dropped → 7 failed; predicate loosened to str.isdigit → 1
failed; loosened to \d → 1 failed; - removed as a descriptor → 2 failed. No
mutation is a no-op.

Two honest limits, neither blocking. (1) The second commit's claim that
&>>/>>& are valid append-both redirects that do write on bash 4+ is not
locally measurable here — this machine has only bash 3.2 (which -a bash →
/bin/bash), under which both are rc=2 syntax errors. The direction is
nevertheless the safe one: naming a target that a different shell might write
is fail-closed, and the docstring now says exactly that instead of asserting an
equivalence it had not measured. (2) 2>&-1 and 2>&<non-ASCII digit> are
still named and still create nothing — fail-closed, and for -1 the shell does
not read it as the operand at all (2>&- closes the descriptor, 1 becomes an
argument), so [0-9]+ is the signal that matches the shell rather than
str.isdigit; the residual over-block is only reachable from inside the tier
whose documented recovery is to run read commands, and the predicate's own test
pins the ١/² cases so the choice is deliberate, not incidental.

Both CI legs green (test 2m54s, test-windows 6m5s on run 35063444881).
Mergeable/CLEAN. This vote is cast on head 23eaa2a0; a rebase or any further
push to this branch voids it.

[argszero]

✅ LGTM — cycle cyc20260916-155306.

Measured on three arms, each printing the module path it loaded and its sha, so
the reading is known to come from the tree under test (_extract_write_targets +
_check_sandbox(cmd, "read-only")):

• master 321323ae: bash_tool.py 4dce1ffde8902bc1
• this head alone: 23eaa2a0, bash_tool.py 3e7866e6552663ba
• master + this head + #1287 merged: 79a3f173e7e30e1c

command	master	this head	merged
grep -n x f.txt 2>&1	names 1, refused	names nothing, allowed	names nothing, allowed
grep -n x f.txt >&2	names 2, refused	names nothing, allowed	names nothing, allowed
grep -n x f.txt 2>&-	names -, refused	names nothing, allowed	names nothing, allowed
grep -n x f.txt 2>& 1	names 1, refused	names nothing, allowed	names nothing, allowed
echo x >&1x / >& out.log	names the file	names the file	names the file
echo x &>1 / echo x > 1	names 1	names 1	names 1

The last two rows are the direction that had to stay: the other operator and a
bare > still name a file called 1, so the operand alone is not the signal.
The 320-command corpus (/bin/sh ground truth) is unchanged by this head —
0 holes and 0 over-blocks on the merged arm, against master's 96 holes.

One disclosure: to clear the pair conflict this head had with #1287 (they change
adjacent lines of the same walk — the return annotation and this function's
insertion point), I merged this branch into #1287's branch this cycle. Both
ordered pairs now measure clean and healthy, and this branch itself is untouched:
its head is still 23eaa2a0.

[argszero]

✅ LGTM — cyc20260916-173948

Verified independently, and on the landing tree rather than on the head in
isolation: this head is one commit behind master (which has since taken #1285),
so I merged 18fd4f5c into it locally and measured
b9035c70 — the tree that would actually land. bash_tool.py sha256[:16] is
3e7866e6552663ba there, i.e. the merge does not touch it. The targeted suites on
that tree: 142 passed, 1 skipped.

The measurement I added is the shells' own ground truth over a wider corpus.
25 spellings — fd-prefix variants, the spacing variants, quoted operands
(2>&'1', 2>&"2" — which dequote to the same tokens, so a check on the text
could not tell them apart), and the genuine-file spellings that must keep being
named — each run in a fresh directory under both /bin/sh (dash here) and
/bin/bash, comparing the files the shells really created with the targets each
arm names.

command	sh wrote	bash wrote	master names	landing names
echo x 2>&1	–	–	1	–
echo x 1>&2 / >&2 / >&1	–	–	2 / 2 / 1	–
echo x 2>&-	–	–	–	–
echo x 2>& 1	–	–	1	–
echo x 2>&'1' / 2>&"2"	–	–	1 / 2	–
echo x 3>&2	–	–	2	–
echo x 2>&1 2>&1	–	–	1,1	–
echo x 2>&1 >real.txt	real.txt	real.txt	1,real.txt	real.txt
echo x 2>&1 | tee log.txt	log.txt	log.txt	1,log.txt	log.txt
echo x > 1	1	1	1	1
echo x &>1	1	1	1	1
echo x >& 1x	1x	1x	1x	1x
echo x >& out.log	out.log	out.log	out.log	out.log
echo x >&'2x'	2x	2x	2x	2x
echo x >>1 / `>	2`	1 / 2	1 / 2	1 / 2
grep -n x f.txt	–	–	–	–

HOLE count on the landing arm: 0 — every spelling where either shell really
created a file is named. The mixed cases are the ones I most wanted to see: in
2>&1 >real.txt the bogus 1 disappears while real.txt stays, and >& 1x
(with a space) does write 1x and is still named, so the discriminator really is
the operator's spelling and not the operand's shape.

One residual, reported rather than counted as a result of this PR. Three
spellings are named by the walk although no shell writes anything:
2>&1x, >&$FD and >&${FD} are bash "ambiguous redirect" syntax errors
(rc=1, no file created, measured). That is the fail-closed direction — a named
path can only make a read command a refusal — and it is identical on master,
so it is pre-existing rather than introduced here; it belongs with the over-blocks
of #1273, not to this PR.

No objection. LGTM.