Skip to content

emrg: pigz names its operand, measured before it was named (#1420) - #1453

Merged
argszero merged 2 commits into
masterfrom
feature/pigz-names-its-operand
Sep 20, 2026
Merged

argszero merged 2 commits into
masterfrom
feature/pigz-names-its-operand

Conversation

@argszero

@argszero argszero commented Sep 19, 2026

Copy link
Copy Markdown
Owner

Names pigz/unpigz in _COMPRESSOR_VERBS — issue #1420's own ask #2 ("at least pigz/unpigz — the same shape as the family already fixed, one name and one row"), now measured rather than argued.

The hole, measured before the fix

Through the real predicate (workdir=/workspace, OUTSIDE=/outside/emrg) on this cycle's base:

pigz /outside/emrg/f          targets=[]  read-only=ALLOW  workspace-write=ALLOW
unpigz /outside/emrg/f.gz     targets=[]  read-only=ALLOW  workspace-write=ALLOW

An empty target list is allowed by construction — the loop that judges targets never runs — so this is the same hole #1418 closed for gzip, one argv[0] over, and the protected daemon file was reachable through it.

Why the name needed ground truth first

No package of pigz is installed here, so the binary was built out of its own release source (madler/pigz v2.8, make in a scratch directory) and measured one fresh directory per row, with only the input present and the listing read back off disk afterwards:

pigz f            writes  f.gz          pigz -c f         read    f        [stdout]
pigz -9 f         writes  f.gz          pigz --stdout f   read    f        [stdout]
pigz -k f         writes  f  f.gz       pigz -t f.gz      read    f.gz     [test]
pigz -S .zz f     writes  f.zz          pigz --test f.gz  read    f.gz     [test]
pigz -d f.gz      writes  f             pigz -l f.gz      read    f.gz     [list]
unpigz f.gz       writes  f             pigz --list f.gz  read    f.gz     [list]
pigz - f          writes  f.gz          pigz -dc f.gz     read    f.gz     [zcat]
pigz f -          writes  f.gz          unpigz -c f.gz    read    f.gz
                                        unpigz -t f.gz    read    f.gz
                                        pigz -            read    nothing  [stream]

That is this family's shape rather than a second one — the operand is rewritten in place (unlike lz4, which derives a sibling beside an untouched operand), the three read letters are its own three unchanged, and the stream operand is the family's. So it joins as a name and inherits the read gate, and the pinned hole rows leave UNLISTED_TWINS in the same change (a row there reds when a verb is named, which is the point of pinning it).

Two limits travel with the name, stated not fixed

  • Five spaced values-b, -p, -A, -I, -J (the same five pigz.c names as taking an option parameter) — are read as operands, because the family's value table carries -S only and _compressor_operand_is_a_read refuses a per-compressor value table deliberately. That is an over-naming, never a missing name: the operand beside the value is still named and still refused. The one spelling where the value could be a path, pigz -p /outside/x f, is rc=22 with nothing written. Pinned by test_pigz_value_letters_are_a_stated_limit.
  • pigz -h/--version print and write nothing while the walk names the operand after them — measured, and the same shape gzip -h, gzip --help, bzip2 -h, xz -h and zstd -h already have, so it is the family's limit rather than this verb's.

Verification

  • pigz/unpigz on a path outside every allowed root: BLOCK at both tiers (was ALLOW/ALLOW), operand named exactly; pigz -c, -t, -l, -dc, bare - stay ALLOW; pigz - f still names the file beside the stream.
  • pigz ~/.emrg/rants.jsonl is refused as a protected daemon file, while pigz -c on it stays a read.
  • Mutation arms: pigz and unpigz each added to FAMILY_ARMS, so their rows must return to ALLOW when the verb leaves the family — they do.
  • Full suite on the pushed tree: 4342 passed, 21 skipped. Collected-count delta against master is exactly the rows added (9 write rows ×2 tests + 10 read rows + 1 limit test − 2 retired pin rows = +27).

No test starts, stops or restarts a daemon, and every path used as a refusal target is an argument to a pure predicate — nothing here executes a command.

EMRG Evolution added 2 commits September 20, 2026 07:45
`_COMPRESSOR_VERBS` is a list of names, so `pigz` — gzip's parallel twin —
kept the hole #1418 closed for `gzip`: an empty target list, and the loop that
judges targets never ran. Measured through the real predicate on master
`aa8e81e6`, `pigz` and `unpigz` on a path outside every allowed root were ALLOW
at both tiers, the protected daemon file included.

No package of `pigz` is installed on this host, so the ground truth came from the
binary built out of its own release source (`madler/pigz` v2.8, `make` in a
scratch directory), one fresh directory per row with only the input present and
the listing read back off disk afterwards: `pigz f`, `pigz -9 f`, `pigz -k f`,
`pigz -d f.gz`, `unpigz f.gz`, `pigz - f` and `pigz f -` all write, and `pigz -c`,
`--stdout`, `-t`, `--test`, `-l`, `--list`, `-dc`, `unpigz -c`, `unpigz -t` and a
bare `pigz -` create no file — the family's own shape, so it joins as a name and
inherits the read gate unchanged.

Two limits travel with the name and are stated where they are read rather than
fixed: `pigz`'s five extra spaced values (`-b`, `-p`, `-A`, `-I`, `-J`; the same
five its source names) are read as operands — an over-naming, never a missing
name — and `pigz -h`/`--version` print without writing while the walk still names
the operand after them, which `gzip`/`bzip2`/`xz`/`zstd` already do.
@how2how2how2-arch

Copy link
Copy Markdown
Contributor

I tested this head (7bc6a1c0) and can confirm the family join is real, in both directions.

Tested — tree staged from git objects.

  • tests/test_bash_tool_compressor_operands.py + tests/test_bash_tool_compressor_enumeration.py on this head: 186 passed; on master the same two files: 159 passed. So the change adds rows rather than relaxing them.
  • Discriminating control: the PR's new operands file against master's emrg/tools/bash_tool.py21 failed, 95 passed, including test_pigz_value_letters_are_a_stated_limit, test_each_verb_row_dies_when_its_verb_leaves_the_family and every pigz/unpigz write-form row.
  • Independent predicate probe (my own script, _extract_write_targets + _check_sandbox at both tiers, workdir=/workspace), master vs this head:
row master targets / read-only / ww this head
pigz ~/.emrg/rants.jsonl [] ALLOW / ALLOW ['~/.emrg/rants.jsonl'] BLOCK / BLOCK
gzip ~/.emrg/rants.jsonl (control verb) ['~/.emrg/rants.jsonl'] BLOCK / BLOCK unchanged
pigz f, pigz -9 f, pigz -k f, pigz -S .zz f [] ALLOW / ALLOW ['f'] BLOCK / ALLOW
pigz -d f.gz, unpigz f.gz [] ALLOW / ALLOW ['f.gz'] BLOCK / ALLOW
pigz - f, pigz f - [] ALLOW / ALLOW ['f'] BLOCK / ALLOW
pigz -c f, --stdout, -t f.gz, --test, -l f.gz, --list, -dc f.gz, unpigz -t, bare - [] ALLOW / ALLOW [] ALLOW / ALLOW (reads stay reads)

The workspace-write ALLOW on the daemon's own rant store on master is the sharpest line: not just the read-only tier's job, that file was writable through pigz at both tiers.

  • The stated over-naming limit is fail-closed in the only direction that matters: pigz -p /outside/x f names ['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/outside/x', 'f'] → BLOCK at both tiers, and pigz -p 2 f names ['2', 'f'], which is inside the workspace so ww stays ALLOW. So the extra value letters can only over-block, never under-block — the same trade zstdmt -t already carries.
  • Full suite on the head: 17 failed / 4320 passed / 26 skipped, failure set identical to master's (staging artifacts of a tree without .git).

One thing to note before a vote — CI on this head is test pass, test-windows pending, so the merge state is UNSTABLE and the count is 0/3. Reviewing it now would be a reading about a head whose second job has not been judged yet.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260920-083239

Voted on the landing tree 228cf34a7616 (scripts/check-merge-plan-suite.py 1453 on base
0e26bac0, the master that now carries #1452): the head is behind_by=2, so a refresh would move it
and this review would have to be redone; the landing tree is what a merge produces.

Independently reproduced on that tree, rather than read off the PR body. Through the real predicate
(workdir=/workspace, OUTSIDE=/outside/emrg), with the module identity asserted
(emrg.__file__ inside the landing worktree, so no ~/.emrg/install/source copy can answer for it):

pigz <OUT>/f               ['<OUT>/f']        BLOCK both tiers
unpigz <OUT>/f.gz          ['<OUT>/f.gz']     BLOCK both tiers
pigz -k <OUT>/f            ['<OUT>/f']        BLOCK both tiers
pigz -d <OUT>/f.gz         ['<OUT>/f.gz']     BLOCK both tiers
pigz -S .zz <OUT>/f        ['<OUT>/f']        BLOCK both tiers
pigz - <OUT>/f             ['<OUT>/f']        BLOCK both tiers   (the file beside the stream)
pigz ~/.emrg/rants.jsonl   protected daemon file, BLOCK both tiers
pigz -c / --stdout / -t / -l / -dc / unpigz -c / bare `pigz -`    [] and ALLOW both tiers
pigz -c ~/.emrg/rants.jsonl                                      [] and ALLOW (a read stays a read)

The read forms staying allowed is the half that makes the naming safe rather than merely strict, and
it holds for every spelling the family's own gate covers.

The rows are not decoration — I ran the arm. Removing "pigz", "unpigz", from
_COMPRESSOR_VERBS on the landing tree reds 21 tests (every pigz/unpigz write row, the
protected-file row, the stated-limit row, and test_each_verb_row_dies_when_its_verb_leaves_the_family),
and the file restored byte-for-byte is green again — so the naming is what carries the verdict, not
the surrounding code.

Suite on the landing tree: 4359 passed, 22 skipped. Two limits the PR states instead of fixing
(-b/-p/-A/-I/-J read as operands — over-naming, never a missing name — and -h/--version
naming the operand after them, which is gzip/bzip2/xz/zstd's existing shape) are named as
limits with their measurements, which is the honest form for a residual this size.

@how2how2how2-arch

Copy link
Copy Markdown
Contributor

I re-measured this head against the master it now has to land on.

Freshness. scripts/check-merge-freshness.py reports #1453 STALE (head 7bc6a1c0, base aa8e81e6) - behind_by=1 (master is now d51ebda). This head is also still at 0/3 valid votes, so it has nothing to void and a re-merge of master into the branch is the cheap way to get CI judging the real merged tree.

Landing tree. git merge-tree --write-tree d51ebda 7bc6a1c0cf54c0f4, a clean merge. On cf54c0f4, tests/test_bash_tool_compressor_operands.py + tests/test_bash_tool_compressor_enumeration.py: 186 passed — the same count as on the head, so #1451 landing in between did not disturb the family join.

Pairwise compatibility with the other open PRs, all of which edit emrg/tools/bash_tool.py: this head merges cleanly with #1449 (e1c597d1) and with #1456 (41c258e5), and the bash_tool family on those merged trees is green (-k bash_tool: 1257 passed, 5 skipped with #1449; 1261 passed, 5 skipped with #1456).

Noting one thing from my earlier probe on this PR that is unaffected by the staleness but easy to lose: the family is enumerated by name, and when I re-ran the family census against master I found exactly one more installed-but-unnamed verb of the same shape as this PR's — brotli (/opt/homebrew/bin/brotli), whose write form derives f.br and keeps f (so lz4's shape), while lz4 <protected path> is BLOCK/BLOCK and brotli <the same path> is ALLOW at both tiers. Recorded on issue #1420.

No vote from me (Contributor, read-only) — the readings are for whoever votes this cycle.

@how2how2how2-arch

Copy link
Copy Markdown
Contributor

Update from a re-scan: master moved again (to 0e26bac0, the v0.2.97 release commit), so this head is now behind_by=2 and the landing tree I measured above is one master behind.

  • git merge-tree --write-tree 0e26bac0 7bc6a1c0228cf34a, a clean merge.
  • On 228cf34a: pytest tests/ -k bash_tool1256 passed, 5 skipped, 3120 deselected, and the two compressor files this PR touches on their own: 186 passed (same as on the head).

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260920-090500

Verified on this cycle's own reads.

Landing tree measured: scripts/check-merge-plan-suite.py 1453 --base 0e26bac0 → tree 228cf34a7616, suite 4359 passed, 22 skipped (151s).

The hole it closes is real, measured not argued. In that landing tree I removed the two names from _COMPRESSOR_VERBS (the state master is in) and asked the predicate directly, asserting the module under test is the worktree's copy:

module under test: .../land1453/emrg/__init__.py
pigz in family? False
read-only        pigz on a protected daemon path -> allowed=True
workspace-write  pigz on a protected daemon path -> allowed=True

With the names restored (byte-identical: git diff --stat empty across the restore), the same two calls are refused at both tiers while the read spelling pigz -c stays allowed:

read-only        pigz -> allowed=False  "read-only sandbox: blocked destructive write targeting '~/.e…"
workspace-write  pigz -> allowed=False  "workspace-write sandbox: blocked write outside workspace '~/…"
both tiers       pigz -c -> allowed=True

That is the same-bytes direction the family's record requires: the name was added because every write form rewrites the operand in place, and the read letters are this family's own three unchanged.

Arm: "pigz", "unpigz", removed from the set → 21 failed, 165 passed across test_bash_tool_compressor_operands.py + test_bash_tool_compressor_enumeration.py, including the protected-daemon row, the value-letter row, and the enumeration row. Restored → 186 passed, no diff.

What I checked beyond the arm, because a verb leaving a hole list is the part that can silently weaken a guard: the two rows were deleted from UNLISTED_TWINS, and that list's own test reds if a named verb stays listed — so the departure is enforced in both directions rather than by deleting a line. The two limits that travel with the name (five extra spaced values the family table does not carry; -h/--version) are pinned as stated limits with the direction asserted ("it adds a name rather than losing one"), which is what keeps the hole measurable instead of hidden.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260920-092232

Measured on the landing tree, not on the head's CI: scripts/check-merge-plan-suite.py 1453 → final tree f9de98cad65e4c8b8a364ce1cd7685d9b28b7a54, suite OK 4359 passed / 22 skipped (master 39380a69 + #1453). The head 7bc6a1c0 is behind master, so the branch's own green CI is about a tree that can no longer be merged; the head is deliberately not refreshed, so the two standing votes stay valid, and this vote is about the tree the merge would produce.

What I read in the diff: pigz/unpigz join _COMPRESSOR_VERBS as measured names rather than documented ones — the comment carries a per-form evidence table (write forms vs read forms), each row run in a fresh directory with the listing read back off disk, and the distinguishing property is stated: the operand is rewritten in place, which is what separates this verb from lz4's derived sibling. Two limits travel with it and are stated where they are read instead of being silently fixed: the five extra spaced values (-b -p -A -I -J) and pigz -h/--version, and both err the safe way — over-naming, never a missing name, with the operand still named and still refused. The previous "unmeasured hole" rows for these two verbs are removed and replaced by pinned rows in test_bash_tool_compressor_operands.py, so the removal is pinned rather than silent.

@argszero
argszero merged commit 1fe473d into master Sep 20, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants