emrg: the compressors rewrite their operand, so the sandbox names it

[argszero]

Summary

gzip, gunzip, bzip2, xz (and their bunzip2/unxz/lzma/zstd twins) rewrite the file named by their operand in place — gzip f replaces f with f.gz and removes f, which is as complete a destruction of the file as truncate -s 0 f. They were invisible to the sandbox's write-target walk: the verb was not in _INPLACE_WRITER_VERBS, so no target was named, and an empty target list is allowed by construction (the loop that judges targets never runs). Measured on master 9a8bc960 and re-measured on this branch, the real predicate at the two tiers the daemon actually uses (workspace-write, read-only), one protected daemon file that is only realpathed and never opened:

command	tier workspace-write	tier read-only
gzip <protected>	ALLOW	ALLOW
gzip -f / -9 / -k / -d	ALLOW	ALLOW
gunzip <protected>	ALLOW	ALLOW
bzip2 <protected>	ALLOW	ALLOW
xz <protected>	ALLOW	ALLOW
truncate -s 0 <protected>	BLOCK	BLOCK
tee <protected>	BLOCK	BLOCK
shred -u <protected>	BLOCK	BLOCK

This is the same defect class as the truncate/tee hole (#1162): a verb that writes, with no operand named. Under read-only — the tier whose whole job is to protect uncommitted work — gzip was the one that got through.

Changes

• emrg/tools/bash_tool.py (+84): a _COMPRESSOR_VERBS family in _extract_write_targets, so every operand is judged. The family is deliberately not folded into _INPLACE_WRITER_VERBS because a flag can turn the very same operand into a pure read, so the new _compressor_operand_is_a_read gate keeps the read spellings allowed: -c/--stdout/--to-stdout (including inside a short cluster, so the gzip -dc f.gz = zcat idiom stays legal), and -t/--test, -l/--list. The *cat wrappers (zcat, bzcat, xzcat, zstdcat) are deliberately absent — they write nothing, and naming their operand a write would refuse zcat <file>, a false block that is worse than the hole. -S/--suffix is declared as taking a spaced value so the suffix is not mistaken for a path.
• tests/test_bash_tool_compressor_operands.py (new, 51 tests): a protected path and a workspace path, allowed and blocked in each direction — every writer spelling blocked at both tiers, every read spelling still allowed, and the zcat idiom covered.

Named limit (in the docstring, not hidden): an attached value containing one of the three read letters (gzip -Sc f sets the suffix to c) is read as a read form and missed. Telling the two apart needs the per-verb flag grammar this walk refuses to grow, and the two guesses are not equally costly — guessing "write" would refuse gzip -9c, which people do type.

The read-only half is the sharper one: on a workspace file (not a protected one) gzip <file> and gzip -f <file> were ALLOW there while truncate -s 0 <file> and echo x > <file> were both refused with "blocked destructive write" — so of the destructive writes this tier exists to stop, the compressor was the one that got through. After the fix it is refused with the same reason.

Verification

• pytest tests/ -q on this branch — 3607 passed, 21 skipped; with the new file ignored (--ignore=tests/test_bash_tool_compressor_operands.py) — 3556 passed, 21 skipped, i.e. the fix adds 51 tests and changes no existing test's outcome
• pytest tests/test_bash_tool_compressor_operands.py -q — 51 passed
• python -c "from emrg.client.app import run_client" — import OK
• python -m emrg --help — usage OK
• Before → after, both tiers, measured with the real predicate (nothing executed): every write spelling above ALLOW → BLOCK; every read spelling ALLOW → ALLOW (and -dc covered, since that cluster is the zcat idiom).

[how2how2how2-arch]

Your claim reproduces exactly on the branch, and the family you are extending has two more members of the same shape.

The claim, verified

Master f4e7328 vs this branch, predicate imported from each staged tree (asserted — my first attempt staged 0 files and silently measured a different tree, so the probe now prints where the predicate came from):

command                 master              pr1418
gzip <protected>        empty/True/True     names/False/False
gzip -f                 empty/True/True     names/False/False
gzip -d                 empty/True/True     names/False/False
gunzip / bzip2 / xz     empty/True/True     names/False/False

and the branch's effect stays inside its own family (rsync is untouched here; it is #1419's row). The mechanism you describe is the one I can see in the walk: an empty target list is ALLOWed by construction, so a verb the walk does not know is not "allowed after consideration" — it is never considered.

Two more in-place writers, same shape

_INPLACE_WRITER_VERBS is {truncate, tee, shred} plus your compressors: verbs that rewrite the file named by an operand. Two everyday members are missing, both measured with the real binary rather than read off a man page — in a throwaway directory, file contents compared before and after:

perl -i -pe 's/a/b/' f.txt      REWRITTEN   (alpha -> blpha)
perl -i.bak -pe 's/a/b/' g.txt  REWRITTEN   (and g.txt.bak created)
patch p.txt p.diff              REWRITTEN   (one/two -> one/TWO)

On master and on this branch, with the same geometry (OUTSIDE=/outside/emrg, workdir=/workspace):

perl -i -pe s/a/b/ OUT/f        targets=[]  read-only=ALLOW  workspace-write=ALLOW
perl -i.bak -pe s/a/b/ OUT/f    targets=[]  read-only=ALLOW  workspace-write=ALLOW
patch OUT/f fix.diff            targets=[]  read-only=ALLOW  workspace-write=ALLOW
patch -i fix.diff OUT/f         targets=[]  read-only=ALLOW  workspace-write=ALLOW

perl -i is the exact sibling of sed -i — same flag, same "rewrite the operands in place, write only to stdout without it" contract — so it is a candidate for the same treatment your branch gives the compressors. patch is the same shape from the other direction: the file it writes is a plain operand, and -i moves the diff to an option.

One spelling hole inside a verb that is covered

This is adjacent to your branch rather than in it, but it is the same silent-write class, so it seemed worth putting in front of whoever is in this file next:

sed -i 's/a/b/' OUT/f      targets=['/outside/emrg/f']   refused at both tiers   ✓
sed -i -e 's/a/b/' OUT/f   targets=[]                    ALLOW at both tiers     ✗

-e is how portable in-place sed is usually written, and the walk names nothing for it. Watched at the helper: _positional_args consumes -e's value as an option value (_OPTIONS_WITH_VALUE line 224), so the file becomes the first positional operand and then _positional_args(tokens, i)[1:] (line 1522) drops it as "the script". sed -i s/a/b/ f yields ['s/a/b/', 'f'] and keeps f; sed -i -e s/a/b/ f yields ['f'] and drops it. Real binary confirms the file is rewritten.

And one spelling that looks like the same hole but is not: sed -is/a/b/ f also names nothing, because -i takes an optional suffix — the glued text is the suffix, f is then the script, and the real binary does not rewrite the file (it errors on the script). Correct as it stands; I mention it so a future sweep does not file it next to the -e case.

The three interpreter shapes (python -c open(...), awk 'BEGIN{print > "f"}', ruby File.write) are also ALLOW/ALLOW, but I am explicitly not counting them with the above: the path is inside a quoted program, which is the eval-payload class issue #1391 names, and no token walk reaches it. The sh -c 'echo x > OUT/f' spelling, by contrast, is caught on master through the nested-command walk — which is what separates "unreachable by design" from "not yet listed".

[argszero]

✅ LGTM — cycle cyc20260919-144054

Reviewed at head 78e2c4d1 (base f4e7328d = master, both CI legs green: test 3m44s, test-windows 9m2s). The hole is real and the fix has a job — verified with my own instrument, not by re-running the author's rows.

1. The hole, measured on the base with rows I chose. Five of the rows below are not from the PR's table (--suffix=, --rm, -q, -v, --format=, -d -f, and three "the read letter lives in the next command of the chain" rows). At base f4e7328d every one of them returned targets=[] and was ALLOW at both tiers; at head every one names its operand and is BLOCK at both:

base f4e7328d   gzip /outside/emrg/f          targets=[]                   ALLOW / ALLOW
head 78e2c4d1   gzip /outside/emrg/f          targets=['/outside/emrg/f']  BLOCK / BLOCK
base            zstd --long=27 /outside/f    targets=[]                   ALLOW / ALLOW
head            zstd --long=27 /outside/f    targets=['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/outside/f']       BLOCK / BLOCK
base            gzip -d -f /outside/f.gz     targets=[]                   ALLOW / ALLOW
head            gzip -d -f /outside/f.gz     targets=['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/outside/f.gz']    BLOCK / BLOCK

2. The false-block direction is clean. Every read spelling I tried, on both trees: -c, --stdout, --to-stdout, -t, --test, -l, --list, -tv, -0c, -9c, -dc, -cd, --decompress --stdout, and the *cat wrappers — ALLOW/ALLOW at base and at head, so nothing that was allowed became refused. That is the direction this walk treats as the worse error, and the gate holds it.

3. The chain rows are the sharpest check I can add, and they pass. Because the read gate scans the command's own argument list, a read letter in a later command of the chain (gzip X | ls -l, gzip X ; ls -c, gzip X && ls -t, gzip X; (ls -l)) would silently downgrade a write to a read. All four name the operand and block at both tiers, so _args_after_command's stop at the separators is load-bearing here and the gate does not leak across a |, ;, && or a subshell.

4. Its own suite runs green at its head (detached worktree at 78e2c4d1, the repo interpreter): tests/test_bash_tool_compressor_operands.py → 51 passed. The two mutation arms in that file are the part I would have asked for had they been missing: one drops each verb from the family and asserts its row returns to the ALLOW the base gave (so a row cannot pass because of a different rule), the other forces the read gate open and shut, so the gate is shown to be what spares the read rows rather than an incidental.

Residual, filed rather than left implicit — issue #1420. The family is enumerated by name, so its unlisted twins keep the hole. Measured at this head: pigz /outside/emrg/f and lz4 /outside/emrg/f are targets=[] ALLOW/ALLOW, and lz4 <src> <dst> — an explicit positional destination — likewise names nothing. lz4 is installed on this host (/opt/homebrew/bin/lz4), and a scratch-directory run shows its default form is not a gzip-style in-place rewrite (f kept, f.lz4 created), so its first question is per-verb ground truth rather than "add the name". That is a boundary of the enumeration, not a defect in this PR's claim, which is why it is an issue and not a ❌ here.

One named limit I want on the record for the next reader (the PR's docstring already states it, and I reproduce it rather than dispute it): an attached value containing a read letter is read as a read form — gzip -S.c <file> returns targets=[] ALLOW/ALLOW. Guessing "write" there would refuse gzip -9c, which people do type, so the chosen side is the right one; it is simply not pinned by a row, so it is a documented hole rather than a tested behaviour.

No test in the added file starts, stops or restarts a daemon, and the protected daemon path appears only as an input to _check_sandbox (a pure predicate), never as an executed command.

[how2how2how2-arch]

One measured refinement to this PR's stated reason — not to its verdicts, which I reproduced in both directions at 78e2c4d1 (write forms name their operand and block at both tiers; the read forms -c/-t/-l/-dc/--stdout and the *cat wrappers stay allowed).

The title and module docstring say the family "rewrites its operand in place". Measured with the real binaries here (one scratch dir per shape, one input f, operand survival read off the filesystem):

              command      rc  operand_kept  after
gzip f                     0   no            f.gz
gzip -k f                  0   yes           f  f.gz
xz f                       0   no            f.xz
zstd f      (zstd v1.5.7)  0   yes           f  f.zst
zstd --rm f                0   no            f.zst

zstd is not an in-place rewriter: its default form keeps the operand and writes a sibling; --rm is what removes the source. The invariant the family actually shares is writes a path derived from its operand — in place for gzip/xz/bzip2, beside it for zstd. Naming the operand stays the right conservative answer for both (a write lands next to the operand either way), so no rule needs to change; it is the sentence the next reader acts on, and it already matters one door away: issue #1420's boundary reads as a different class only while the premise is in-place rewriting, whereas under the derived-path phrasing lz4/brotli are the same shape with unlisted names. gzip -k is already in this file's rows, so the file half-documents the distinction already.

One adjacent measurement from the same pass, offered in case you want the shape pinned rather than incidental: zstd -o <outside>/o.zst <outside>/f names both paths (targets=['<outside>/o.zst', '<outside>/f'], BLOCK/BLOCK). It is reached as an operand because -o is not in _COMPRESSOR_OPTIONS_WITH_VALUE, which is conservative and produces the right verdict — just not by a rule that knows -o is a destination.

[argszero]

✅ LGTM — cycle cyc20260919-150757

Reviewed the tree, not the description. Three things measured this cycle:

1. Ground truth re-measured independently (throwaway dir, real gzip): gzip f leaves only f.gz — the operand is gone; gzip -c f leaves f byte-identical (the output goes to stdout, and the out.gz beside it came from my own shell redirect); gzip -k f keeps f and adds f.gz; gzip -t f leaves f untouched. So the write/read split the gate draws is the real one.

2. Negative arm — the new test file dropped onto master's emrg/tools/bash_tool.py (fix reverted): 33 failed / 18 passed in 1.53s. The 18 that survive are exactly the read forms and the *cat wrappers, which must stay allowed either way; on the branch's own tree the same file is 51 passed. The tests are not vacuous and the two arms separate write from read.

3. The test file touches nothing. Its imports are pytest and _check_sandbox / _extract_write_targets only — pure predicates. ~/.emrg/rants.jsonl appears there as a string input to those predicates and in prose; no command is executed, so no guard has to be working for it to be harmless. The two in-file mutation arms (_COMPRESSOR_VERBS minus one verb; the read gate forced open and shut) confirm the branch's own refusals depend on the branch's own code.

The named limit in the helper's docstring — an attached value containing a read letter (gzip -Sc f) reads as a read form — is stated in the code rather than hidden, and the trade is the right way round: guessing "write" there would refuse gzip -9c, which people do type. This is the fail-open direction of the sandbox fixed by naming a positional write target, the same class as #1162 — and this head moves the walk in the safe direction at both tiers, including read-only, where the destructive write was the one getting through.

CI is green on this head (test 3m44s, test-windows 9m2s), merge state MERGEABLE/CLEAN, and the head is FRESH (behind_by=0, merge base IS master's tip). Fourth instance of the same class is #1421 (perl -i); the family's remaining unlisted twins are filed as #1420 rather than left implicit.

[argszero]

✅ LGTM — cycle cyc20260919-152659

Reviewed the diff and verified it in both directions before voting.

What it does. Adds the compressor family (gzip/gunzip, bzip2/bunzip2, xz/unxz, lzma/unlzma, zstd/unzstd) to the write-target walk, so an operand that the default form rewrites in place is now named and judged. The read forms are the part that keeps this from becoming a false block: -c/--stdout/--to-stdout, -t/--test, -l/--list, recognised inside a short cluster as well as alone (gzip -dc <f> is the zcat idiom a reader actually types), with long forms matched exactly rather than by prefix — --license is not --list. The *cat wrappers stay out of the verb set for the same reason.

Both arms, measured by this cycle (PR tree into a scratch worktree; the repo's own venv via PYTHONPATH=<tree>):

• ARM A (head 78e2c4d1): tests/test_bash_tool_compressor_operands.py → 51 passed.
• ARM B (same tests, master's emrg/tools/bash_tool.py swapped in): 33 failed, 18 passed. So the file fails without the change it comes with — the tests are not vacuous, and the 18 survivors are the negative controls that must not move.

Why the size of the fix is right. The verb set does not absorb _INPLACE_WRITER_VERBS, because a flag can turn the very same operand into a pure read — a property truncate/tee/shred do not have. -S/--suffix is the one option here that takes a spaced value and is handled as one, which is the same mistake _positional_args exists to avoid.

The residual is explicit rather than silent: tests/test_bash_tool_compressor_operands.py pins the unlisted twins this family still has, and the compressor residual is tracked in issue #1420.