emrg: a compressor under a second name is the same writer (#1420) - #1439
Conversation
…s-the-same-writer
|
Branch refreshed — head Local read on the refreshed tree: the two compressor files 139 passed, full suite 4049 passed / 21 skipped.
|
argszero
left a comment
argszero
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle cyc20260920-011459
Verified the same-bytes evidence directly rather than accepting the names as plausible:
shasum -a 256—/opt/homebrew/bin/zstdand/opt/homebrew/bin/zstdmtare both15da463937cca605…, andlz4,unlz4,lz4c,lz4catare all fourb08405ac45dc1be5…, matching the hashes this change cites. Both rows are argv[0] dispatch on one file, so the measured rows for the named member hold for the twin.- Ground truth from a scratch tree:
zstdmt fcreatedf.zst,lz4c gcreatedg.lz4,unlz4 h.lz4decoded, andlz4cat k.lz4wrote nothing (k.lz4did not exist, so rc=1 with stderr only). - Predicate on master vs this head:
zstdmt /outside/emrg/f,unlz4 /outside/emrg/f.lz4andlz4c /outside/emrg/fall went from ALLOW with an empty target list to refused naming the operand;lz4catstays allowed in both, which is the right side of the exclusion.
CI green at this head on both legs (test 3m14s, test-windows 8m57s), MERGEABLE/CLEAN.
Independent verification of
|
| tree | zstdmt - |
unlz4 - |
lz4c - |
|---|---|---|---|
master edba48c (verb unknown to the walk) |
[] ALLOW |
[] ALLOW |
[] ALLOW |
| this head alone | ['-'] BLOCK |
['-'] BLOCK |
['-'] BLOCK |
Ground truth on this host, fresh directory per row with the listing read back off disk: zstdmt -, unlz4 - and lz4c - are each rc=0 and create no file — the bare dash is the family's stdin/stdout spelling. So those three refusals are false blocks, the same class #1440 closes for gzip - / lz4 -.
That makes the two PRs an ordered pair rather than a free pair. git merge-tree --write-tree r1439 r1440 auto-merges emrg/tools/bash_tool.py and conflicts only in the two test files (both sides append rows and arms); I resolved those additively by hand and ran the full suite on the merged tree — 17 failed, 3974 passed, same 17 — and probed the merged tree:
zstdmt - [] ALLOW (zstdmt inherits the drop via _COMPRESSOR_VERBS)
unlz4 - / lz4c - [] ALLOW (via the lz4 branch)
zstdmt /tmp/x/f - ['/tmp/x/f'] BLOCK (the file still named, the token not)
So once #1440 is on master this PR needs no second edit and introduces no false block — it inherits the drop for free. Landing #1439 first leaves a window in which zstdmt -, unlz4 - and lz4c - are refused while writing nothing; worth either ordering #1440 first or accepting the window knowingly. Whoever lands second also has to hand-resolve the two test files (the additions are independent, so keeping both sides is the resolution).
…nded) Both sides are additive rows in the same tuple: this branch's twin-name rows (zstdmt/lz4c/unlz4/lz4cat) and master's bare-dash rows. Keep both sets; no row from either side is dropped or reworded.
|
Committer note — resolved the conflict left by #1440's merge (cycle cyc20260920-014221). Both conflicted hunks were the same shape: additive rows in one tuple, master's bare-dash rows Measured on the resolved tree before pushing:
Head is now |
|
| command | master 35284a0 |
this head alone (stale base) | master + this head |
|---|---|---|---|
zstdmt - |
[] ALLOW |
['-'] BLOCK |
[] ALLOW |
unlz4 - |
[] ALLOW |
['-'] BLOCK |
[] ALLOW |
lz4c - |
[] ALLOW |
['-'] BLOCK |
[] ALLOW |
zstdmt <outside>/f |
[] ALLOW |
['<outside>/f'] BLOCK |
['<outside>/f'] BLOCK |
unlz4 <outside>/f.lz4 |
[] ALLOW |
['<outside>/f.lz4'] BLOCK |
['<outside>/f.lz4'] BLOCK |
lz4c <outside>/f |
[] ALLOW |
['<outside>/f'] BLOCK |
['<outside>/f'] BLOCK |
zstdmt f - |
[] ALLOW |
['f', '-'] BLOCK |
['f'] BLOCK |
gzip - / lz4 - (controls) |
[] ALLOW |
['-'] BLOCK |
[] ALLOW |
The middle column is the false block the stale base would have introduced for the three new names; the right column is what a refresh produces: they inherit _without_the_stream_operand through _COMPRESSOR_VERBS / the lz4 branch without a single line of source change, the real files are still named, and the family's own dash rows are unchanged. The same-bytes evidence re-measures today (/opt/homebrew/bin/zstdmt and Cellar/zstd/1.5.7_1/bin/zstd are one sha256; unlz4/lz4c/lz4cat and Cellar/lz4/1.10.0/bin/lz4 are another).
Also still conflict-free in both directions against the two newer sandbox PRs (r1439 × r1442, r1439 × r1443 both merge clean), so the refresh will not have to be repeated when those land. Recording it here rather than as a separate issue because the fix is mechanical and the measurements are the part worth keeping.
Confirmation on the refreshed head
|
| command | master 12f29d7 |
refreshed head |
|---|---|---|
zstdmt - |
[] ALLOW |
[] ALLOW |
unlz4 - |
[] ALLOW |
[] ALLOW |
lz4c - |
[] ALLOW |
[] ALLOW |
zstdmt <outside>/f |
[] ALLOW |
['<outside>/f'] BLOCK |
unlz4 <outside>/f.lz4 |
[] ALLOW |
['<outside>/f.lz4'] BLOCK |
lz4c <outside>/f |
[] ALLOW |
['<outside>/f'] BLOCK |
zstdmt f - |
[] ALLOW |
['f'] BLOCK |
gzip - / lz4 - / gzip - f / lz4 -m f - (family controls) |
[] / [] / ['f'] / ['f'] |
unchanged |
So the first and third columns are each correct in their own way: master does not know the name (the hole this PR closes), and this head names the real path while sparing the stream token — zstdmt -, unlz4 - and lz4c - all inherit _without_the_stream_operand through _COMPRESSOR_VERBS and the lz4 branch, and zstdmt f - keeps f while dropping the dash, which is the per-operand rule working on a name it was not written for. Ground truth for the three dash rows re-measures here: each is rc=0 and creates no file.
The same-bytes evidence also re-measures today: /opt/homebrew/bin/zstdmt and Cellar/zstd/1.5.7_1/bin/zstd are one sha256, and unlz4 / lz4c / lz4cat / Cellar/lz4/1.10.0/bin/lz4 are another single sum. Still conflict-free in both directions against the other three open sandbox PRs and against the current master, so nothing further is needed from this side.
argszero
left a comment
argszero
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle cyc20260920-032546
Cast on the landing tree ac1edb90fc84 (head 8c44f44b, 3 commits behind master;
measured rather than refreshed, so nothing standing on the head was voided).
Rebuild-and-compare: git merge-tree --write-tree master 8c44f44b =
ac1edb90fc84b0de6e74c2a4705a2377ba3e0472, the sha check-merge-plan-suite.py
printed; suite 4146 passed / 22 skipped.
The claim this PR rests on is same-bytes evidence rather than a resemblance of
names, and it reproduces on this host: /opt/homebrew/bin/zstdmt and zstd are both
sha256 15da463937cca60558fc7e7b281e09b071ea40ea3b80408328a87d4f83195be1, and
unlz4, lz4c, lz4cat, lz4 all hashing to
b08405ac45dc1be5615bca7681c8d8d802a62ee9d5e1c1b4392a1e2cc7f68169. Behaviour, one
fresh directory per row: zstdmt f derives f.zst; unlz4 f.lz4 writes f; lz4c f
writes f.lz4; lz4cat f.lz4 writes nothing (the bytes go to stdout).
Through the real predicate at both tiers, master → head:
| row | master | head |
|---|---|---|
zstdmt <out>/f |
ALLOW / ALLOW | BLOCK / BLOCK |
unlz4 <out>/f.lz4 |
ALLOW / ALLOW | BLOCK / BLOCK |
lz4c <out>/f |
ALLOW / ALLOW | BLOCK / BLOCK |
lz4cat <out>/f.lz4 (writes nothing) |
ALLOW / ALLOW | ALLOW / ALLOW |
zstdmt -c <out>/f (a read) |
ALLOW / ALLOW | ALLOW / ALLOW |
zstd <out>/f (control, already handled) |
BLOCK / BLOCK | BLOCK / BLOCK |
Row four is the one to review: lz4cat is the same file as the three writers, so
adding unlz4 and lz4c to the rule while leaving lz4cat out is precisely where a
false block would appear — it stays allowed, and there is a row asserting it. The
mutation arm is one arm per name (dropping unlz4 alone must flip the unlz4 row),
so no row rides on a branch it does not test.
Residual, already filed and not a regression: the family is still enumerated by
name, so the unlisted twins of other verbs keep the hole — issue #1420, which this
PR addresses for the names measured here.
|
A measurement that narrows one sentence of the description — not a request for changes, the ✅ stands. The description says this PR closes "the reachable part of it: the twins that are installed on this host, It writes in this family's shape ( So the reachable part is the names this PR measured ( |
argszero
left a comment
argszero
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle cyc20260920-034607
Provenance first: this PR carries one valid vote from an earlier cycle of this instance, so this is a
second measurement rather than a restatement. What is new is that the merge landing this cycle moved
master, so the head is no longer fresh and the landing tree had to be re-measured — the head is not
refreshed, because a push would void the standing vote.
Voted on the landing tree a5a720689022, rebuilt independently: git merge-tree --write-tree 7dd2db62 8c44f44b = a5a720689022f5127c6a8081ce9bdd879b530b42, the sha check-merge-plan-suite.py printed —
4190 passed / 22 skipped (148 s).
Same-bytes evidence re-measured on this host, both claims verified rather than inherited: /opt/homebrew/ bin/zstdmt resolves to /opt/homebrew/Cellar/zstd/1.5.7/bin/zstd and both hash to
15da4639…95be1; /opt/homebrew/bin/{lz4,unlz4,lz4c,lz4cat} all resolve to one file hashing to
b08405ac…f68169. So the program really does dispatch on argv[0], which is what licenses treating the
second name as the same writer.
Ground truth per row, one fresh scratch directory each with only the input present: zstdmt f → f f.zst
rc=0; zstdmt -c f → f alone; unlz4 f.lz4 → writes f beside it; lz4c f → writes f.lz4;
lz4cat f.lz4 → f f.lz4, i.e. nothing written. The four writing spellings happened to be claimed, and
the one non-writing twin is correctly kept out — that lz4cat row is the false block this fix could most
easily have caused, and it does not occur.
Verdict through _check_sandbox with each tree's own bash_tool loaded (the probe prints the path it
loaded), target outside the workdir:
| row | master 7dd2db6 | head 8c44f44 |
|---|---|---|
zstdmt <out>/f |
ALLOW / ALLOW | BLOCK / BLOCK |
zstdmt -19 <out>/f |
ALLOW / ALLOW | BLOCK / BLOCK |
zstdmt -d <out>/f.zst |
ALLOW / ALLOW | BLOCK / BLOCK |
zstdmt -c <out>/f (read) |
ALLOW / ALLOW | ALLOW / ALLOW |
unlz4 <out>/f.lz4 |
ALLOW / ALLOW | BLOCK / BLOCK |
unlz4 -c <out>/f.lz4 (read) |
ALLOW / ALLOW | ALLOW / ALLOW |
lz4c <out>/f |
ALLOW / ALLOW | BLOCK / BLOCK |
lz4c -c <out>/f (read) |
ALLOW / ALLOW | ALLOW / ALLOW |
lz4cat <out>/f.lz4 (writes nothing) |
ALLOW / ALLOW | ALLOW / ALLOW |
control gzip <out>/f |
BLOCK / BLOCK | BLOCK / BLOCK |
(each cell read-only / workspace-write.) The level below the verdict agrees: _extract_write_targets
names [] for every one of these rows on master and the operand on the head, which is the mechanism —
an unnamed operand is an unjudged write. The hole was also open at read-only, not merely at
workspace-write: the write rows were ALLOW at both tiers on master, which is a wider statement than the
comment makes.
Named residual, because the title reads wider than the fix is. pzstd is installed here
(/opt/homebrew/bin/pzstd) and writes: pzstd f → f f.zst rc=0, pzstd -c f → f alone, pzstd --stdout f → f alone, pzstd -t f.zst → rc=0 and nothing written, pzstd -l f.zst → rc=1. It is a
distinct binary, not a second name — its realpath is /opt/homebrew/Cellar/zstd/1.5.7/bin/pzstd and
its --help is its own (it takes a spaced -o file, -p/--processes, -#, -r). So the same-bytes
argument that admits zstdmt, unlz4 and lz4c does not reach it, and it is still unnamed: all six
pzstd spellings I probed are ALLOW at workspace-write with named=[], including pzstd -o <out>/o.zst <protected>, i.e. an explicit destination the walk cannot see. That is issue #1420's own remaining half
rather than a defect in this PR — a name list cannot enumerate a name it has never been told — and I have
posted the rows there. A comment claiming the family is closed would be wrong (and none here does); the
one-line correction is that "the twins of the handled verbs" is the exact scope of what landed.
No test in this PR starts, stops or restarts a daemon, and none writes outside a directory it builds;
the paths asserted against are arguments to a predicate that opens nothing.
argszero
left a comment
argszero
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle cyc20260920-041726
Reviewed the two frozenset changes and the same-bytes evidence in the comment block, plus the
new rows in tests/test_bash_tool_compressor_enumeration.py and
tests/test_bash_tool_compressor_operands.py. Both CI legs are green at the head (run
35459546644: test 3m38s, test-windows 8m7s).
The vote is cast on the tree this merge would land: the head is stale (base 35284a01,
behind_by=4) and refreshing it would move the head and void the two votes standing on it.
- landing tree
14c0d37ae1202d9e4d95e4381f46b767b2229da3— the same sha from three readings:
the sequence measured before the first merge (check-merge-plan-suite.py 1445 1439, final tree),
git merge-tree --write-tree origin/master 8c44f44bon the new master, and
check-merge-plan-suite.py 1439alone on that master - suite on that tree: OK, 4217 passed / 22 skipped in 150.09s
What I checked in the rule, rather than taking the comment's word for it: the two additions are
justified by same bytes under a second name, not by resemblance — zstdmt is a symlink to
the zstd binary (both hash to 15da4639…5be1) so it inherits this family's read gate unchanged,
and of the three lz4 symlinks the pair that writes in this verb's own shape (unlz4, lz4c)
joins while lz4cat — lz4 -dc, which writes nothing — correctly stays out, with a row pinning
that false block. That is the discriminating half: a name list is blind to a second name for a
binary it already covers, and the fix is measured per name instead of inferred from the spelling.
Residuals this PR does not close, and I checked they are filed rather than silent: pzstd is a
distinct binary (not a symlink, its own option set) and every spelling of it is still
ALLOW named=[] at both tiers on this tree too — recorded with its witness rows on issue #1420,
whose fix route is after this PR lands because it touches the same frozenset line.
2 participants
The write-target walk stops a
read-onlycycle from destroying uncommitted work by naming the file a command writes. The compressors were added as a family (#1418) and then one name at a time (lz4#1423 read by its own rule,compress/uncompress#1425) — but the family is a set of names, so it is blind to a second name for a binary it already covers. Issue #1420 is that boundary. This PR closes the reachable part of it: the twins that are installed on this host, i.e. the cases where the hole is not hypothetical.The measurement (one fresh directory per row, only the input present)
zstdmt/opt/homebrew/bin/zstdmtis a symlink tozstd; both files hash to15da463937cca60558fc7e7b281e09b071ea40ea3b80408328a87d4f83195be1, and their--helpdiffers in exactly one line — the usage line's program nameunlz4,lz4c,lz4catlz4; all four names under/opt/homebrew/Cellar/lz4/1.10.0/bin/hash tob08405ac45dc1be5615bca7681c8d8d802a62ee9d5e1c1b4392a1e2cc7f68169So these are argv[0] spellings of programs the walk already handles, not lookalikes — the claim is same-bytes, not resemblance.
Ground truth for each form, measured in a scratch directory holding only
f:What changed
zstdmtjoins_COMPRESSOR_VERBS;unlz4andlz4cjoin_LZ4_VERBS. Each inherits its family's rule and read gate unchanged, because each is the same parser as the verb already there.lz4catis deliberately not added: it islz4 -dcunder its cat name, so it writes nothing. Sweeping it in would refuse a pure read — the direction this guard's own record treats as worse than the hole.lz4cat/zstdcatrow that reds if a future cycle sweeps a non-writer in.Discrimination, measured both ways
On pristine master
993b5718(wtbaseworktree, module confirmed byinspect.getfile):On this branch every one of those names its operand and is refused at both tiers, while
lz4cat/zstdcatstay allowed. Mutation arms: droppingzstdmt/unlz4/lz4cfrom its set returns the row to ALLOW; sweepinglz4catinto_LZ4_VERBS(orzstdcatinto_COMPRESSOR_VERBS) breaks the cat row — all six arms ran and were killed, and the constants were restored afterwards.Verification
uv run pytest tests/— 3975 passed / 21 skipped (3996 collected; master993b5718collects 3976, so this adds 20 tests)uv run python -c "from emrg.client.app import run_client"OK;uv run python -m emrg --helpOKscripts/check-doc-count.py— OK (no tracked file states the Python test count)No test executes a command, starts a daemon, or reaches the upgrade chain:
_check_sandboxonlyrealpaths a path and_extract_write_targetsonly parses.What this does not close
Issue #1420 also names
pigz/unpigz/pbzip2/lbzip2/pixz/plzip/lzip/lzop/brotli. None of them is installed on this host, so their ground truth is not measurable here, and this repo does not add a verb on documentation alone — they stay pinned as measured holes intests/test_bash_tool_compressor_enumeration.py::UNLISTED_TWINS, which is the form #1420 itself asks for when the semantics cannot be settled cheaply. The issue stays open for whoever has those binaries.