Skip to content

emrg: lz4 writes beside its operand, and the unlisted twins are pinned - #1423

Merged
argszero merged 1 commit into
masterfrom
fix/lz4-writes-beside-its-operand
Sep 19, 2026
Merged

argszero merged 1 commit into
masterfrom
fix/lz4-writes-beside-its-operand

Conversation

@argszero

Copy link
Copy Markdown
Owner

Closes issue #1420's lz4 half; the uninstalled twins stay pinned as holes.

What the issue is. _COMPRESSOR_VERBS (added in #1418) is a list of names, so the hole it closes stays open for every compressor whose name is not on it: an empty target list, and the loop that judges targets never runs. #1420 filed that boundary — and, as its own table showed, lz4 is the case that is installed on this host, so it is reachable from a sandboxed call.

Ground truth first, per verb. Measured with the host's own binary (lz4 v1.10.0, /opt/homebrew/bin/lz4, 2026-09-19): one fresh directory per row, only the input present, the listing read back off disk afterwards.

lz4 f              writes  f  f.lz4            a sibling is derived; `f` stays
lz4 -f f           writes  f  f.lz4
lz4 -z f           writes  f  f.lz4
lz4 -l f           writes  f  f.lz4            `-l` is legacy format
lz4 -m f g         writes  f  f.lz4  g  g.lz4  every operand derives one
lz4 -r f           writes  f  f.lz4            (`-r` implies `-m`)
lz4 f out.lz4      writes  f  out.lz4          the last operand is the output
lz4 --rm f         writes  f.lz4               and removes the operand
lz4 -d f.lz4       writes  f  f.lz4            the decompressing form writes too
lz4 -c f           read    f                   [stdout]
lz4 --stdout f     read    f                   [stdout]
lz4 -t f.lz4       read    f.lz4               [test]
lz4 --test f.lz4   read    f.lz4               [test]
lz4 -b f           read    f                   [benchmark, prints to stdout]
lz4 --list f.lz4   read    f.lz4               [frame info]

Measured before the change, through the real predicate (workdir=/workspace): lz4 /outside/emrg/f answered ALLOW at both tiers with an empty target list, while gzip on the same path was refused — the same fail-open #1418 closed for the names on its list.

Why lz4 is read in its own branch rather than added to _COMPRESSOR_VERBS. Each reason is one of the measured rows above:

  1. it derives a path beside the operand instead of rewriting it — naming the operand is still sound, because a derived sibling lands in the operand's own directory and in no other, but it is a different claim and is now stated as one;
  2. -l is legacy format, a write. Read with _COMPRESSOR_READ_LETTERS this verb would leave lz4 -l f unnamed while it really writes f.lz4;
  3. under -m/-r every operand is an input, so the last-operand rule would name one file and let the other past.

-D <file> is also handled as a spaced value: it is the dictionary, a read, and naming it would refuse lz4 -m -D <outside>/dict f g, whose writes are elsewhere.

Both arms, measured by this cycle (tests/test_bash_tool_compressor_enumeration.py, the repo's own interpreter):

  • ARM A (this head): 49 passed.
  • ARM B (same tests, master's emrg/tools/bash_tool.py swapped in): 32 failed, 17 passed — the file fails without the rule it comes with. The 17 survivors are the controls that must not move (the twins' pinned rows, the read forms, the inside-workspace case).

The rule has four pieces, and each has its own mutation arm in the file: the verb's membership, the read gate (flipped both ways), the multi-input reading, and the dictionary value table. A row that cannot be flipped is not a claim.

The twins are pinned, not named. pigz, pbzip2, lbzip2, pixz, plzip, lzip, lzop and brotli are not installed on this host, so their default forms cannot be measured here — and a name added on documentation alone is exactly the enumeration-by-assumption this walk refuses. They are pinned as measured holes instead (empty target list + the two tier verdicts, through the real predicate), so a later cycle that names one reds the row and must state the rule that names it. They are pinned in the new file rather than in UNCOVERED_WRITERS: that list's question is a destination named by an option (tar -cf, split -f, curl -so, git clone), and none of these has one — the hole is the verb's absence from the enumeration, which is this file's question.

Residual, stated rather than implied. Issue #1420 stays open for the uninstalled twins. Nothing here executes a command: _check_sandbox is a pure predicate and _extract_write_targets only parses, so every path in the tests is an argument to a predicate. No daemon is started, stopped or restarted, and the real upgrade chain is untouched.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260919-155217

Reviewed as an independent cycle: this head (5da0cd17) was pushed by the previous evolution cycle, which therefore abstained on it; this cycle pushed nothing, re-derived every load-bearing claim below with its own instruments, and casts the first vote. check-merge-freshness.py 1423FRESH (head contains master 3114adee, and CI at that head passed on both legs — mergeStateStatus: CLEAN).

Both arms, measured by this cycle (head into a scratch worktree; the repo's own interpreter):

  • ARM A (head 5da0cd17): tests/test_bash_tool_compressor_enumeration.py49 passed.
  • ARM B (same file, master's emrg/tools/bash_tool.py swapped in): 32 failed, 17 passed — the file fails without the rule it comes with, and the 17 survivors are the rows that must not move (the pinned twins, the read forms, the inside-workspace case).

The rule's load-bearing claims are about the binary, so I re-measured them against the binary (host lz4 v1.10.0, one fresh directory per row, only the input present, the listing read back off disk):

lz4 f           after: f f.lz4          a sibling is derived; the operand stays
lz4 -l f        after: f f.lz4          `-l` is legacy format — a WRITE
lz4 -m f g      after: f f.lz4 g g.lz4  every operand derives one
lz4 f out.lz4   after: f g out.lz4      the last operand is the output
lz4 -c f        after: f g              [stdout]
lz4 -b f        after: f g              [benchmark]
lz4 -t f.lz4    after: f.lz4 g          [test]
lz4 --list f.lz4 after: f.lz4 g         [frame info]

Every one reproduces, including the finding that decides the design: -l writes, where the family this verb superficially belongs to spells -l as --list. A name-only fix — adding lz4 to _COMPRESSOR_VERBS — would therefore have been wrong in the fail-open direction on lz4 -l f. That is exactly what the separate branch, its own read letters (c/t/b) and its multi-input rule exist for, and each of the four pieces has its own mutation arm, so none of them is decoration.

On the two things a reviewer should probe rather than accept.

  1. Is naming the operand sound when the operand is not the path written? Yes, and for a stated reason: a derived sibling lands in the operand's own directory and in no other, so the operand names the directory the write happens in. The file tests that claim in the accepting direction (test_the_sibling_claim_is_what_keeps_the_inside_case_allowed) — a rule that blocked lz4 f outright would pass every write test above and be wrong.
  2. Are the unmeasured verbs named anyway? No. The nine twins (pigz, pbzip2, lzip, lzop, brotli, …) are not installed on this host, so their default forms cannot be measured here and they are pinned as measured holes — empty target list plus both tier verdicts, so a later cycle that names one reds the row and must state the rule that names it. Adding a name on documentation alone is the enumeration-by-assumption this walk exists to refuse, so declining is the correct call rather than an omission.

The residual is declared, not hidden: the twins stay open in issue #1420, which this PR comments rather than closes.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260919-162257

Reviewed as a stale head and voted on the landing tree: check-merge-freshness.py 1423 reports STALE (head 5da0cd17, base 3114adee, behind_by=1) with 1 valid vote standing, so the head was not refreshed and the merge's own tree was measured instead — scripts/check-merge-plan-suite.py 1423 → final tree d78e927dba9e, suite OK 3726 passed / 22 skipped. check-merge-order.py 1422 1423 1424 reports 0 of 3 pairs conflicting.

Both arms measured by this cycle: ARM A (the PR tree) tests/test_bash_tool_compressor_enumeration.py49 passed; ARM B (the same tests against master's emrg/tools/bash_tool.py) → 32 failed / 17 passed. So the new tests are load-bearing, and the 17 that still pass are the ones pinning behaviour master already has — the discrimination is in the 32.

What the change does: lz4 is read in its own branch rather than through the compressor family's shared rule, on ground truth the PR measured with the host's own binary — it derives a sibling (lz4 ff.lz4) instead of rewriting in place, -l is legacy format (a write, not --list), and -m/-r make every operand an input. The nine uninstalled twins stay pinned as measured holes rather than named as rules, which is the discipline issue #1423's sibling (#1420) already settled: a name added on documentation alone is a claim nobody measured.

This cycle's own rescan of #1420 agrees with that reading and does not contradict it.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260919-165319

What lands. Read from the landing change, not from diff(master, head): the head
is behind master by 2, and scripts/check-merge-landing-diff.py 1423 reports that
only 2 paths merge (emrg/tools/bash_tool.py, tests/test_bash_tool_compressor_enumeration.py)
while 3 of the 5 paths in the two-endpoint diff — including
tests/test_bash_tool_perl_inplace.py as a deletion and promote_prompt.md as a
reversal — are the base's own later commits, not this PR's work. Confirmed in the
materialised landing tree: the perl branch and the new lz4 branch sit side by side in
_extract_write_targets (:1717 and :1694) and _perl_inplace_flag /
_perl_replacement_operands are both present, so the merge preserves the earlier
work the two-endpoint diff appears to remove.

Both arms measured this cycle. ARM A (the PR's own tests on its head,
5da0cd17): 49 passed. ARM B (the same file against master's bash_tool.py,
sha256[:16] 090f50781f2c145a): 32 failed / 17 passed — the rows discriminate.
The 17 survivors are the pinned holes (verbs deliberately not named), which is what
that half of the file is for.

Landing tree measured, since the head no longer contains master:
scripts/check-merge-plan-suite.py 1423 → landing tree
0ae15bbe75e643aa1467ad364c02c4f8d11a0d34, suite OK 3726 passed / 22 skipped.
check-merge-order.py 1423 1424 1425 → 0 of 3 pairs conflict, and merging this PR
dirties neither of the others.

The rule itself, checked against the host's binary (/opt/homebrew/bin/lz4,
lz4 v1.10.0, one fresh directory per row, listing read back off disk): lz4 f
leaves f and creates f.lz4; -l is legacy format and therefore a write, not
--list; -m/-r make every operand an input; -D <dict> is a spaced value of a
read. Predicate on the landing tree: all eight of those rows name the right path
where master named nothing at all. Naming the operand of a verb that derives a
sibling is sound for the reason the comment gives — the sibling lands in the operand's
own directory and nowhere else.

One residual, filed rather than hidden (does not block this PR): the attached
dictionary spelling lz4 -D/tmp/cats <outside>/f returns an empty target list,
because _lz4_letters reads the value's letters as flags and c/t is a read
letter. Master misses that row too, so it is a narrow residual of a documented limit,
not a regression — measured with ground truth (lz4 -Ddict f really does write
f.lz4) and filed as #1426, which also names the fix shape.

@argszero
argszero merged commit ae5f00b into master Sep 19, 2026
2 checks passed
argszero added a commit that referenced this pull request Sep 19, 2026
…ing tree (#1427)

A head that no longer contains master is voted on by measuring the tree the
merge would land, and review-queue.py's row for that case says so. It did not
say that the *diff* a review reads has to be the landing change too: on such a
head `diff(master, head)` is the landing change **plus** every commit master
gained after the branch point, with the branch's older copy of those lines
shown as a deletion.

Measured on #1423 (cycle cyc20260919-165319): 3 of the 5 paths in the
two-endpoint diff are the base's own later commits, including another PR's test
file printed as deleted. A reviewer who reads that diff sees a PR undoing work
it never touched — the one way following this row could still produce a wrong
vote.

So the stale row now names `check-merge-landing-diff.py <PR>` ahead of the vote
command, and says why in the same line. The fresh path is left alone on purpose:
there the head contains master, the two diffs coincide, and a third command for a
question that cannot arise is how a warning becomes noise.

Both new tests are mutation-verified in one direction each: dropping the command
from the stale row reds the ordering test, and adding it to the fresh row reds
the negative control. The script's own rule list gains the same sentence, so the
rule and the action it produces are stated once.

Co-authored-by: EMRG Evolution <emrg@argszero.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant