Skip to content

emrg: zip writes its first operand, so the walk names the archive - #1438

Merged
argszero merged 8 commits into
masterfrom
fix/zip-names-its-archive
Sep 19, 2026
Merged

argszero merged 8 commits into
masterfrom
fix/zip-names-its-archive

Conversation

@argszero

Copy link
Copy Markdown
Owner

Closes the last remaining row of #1420: zip writes the archive and the walk named nothing for it.

What was wrong

zip a.zip in.txt creates a.zip — that is what the program is for — and the sandbox walk named no target, so both tiers allowed it. An empty target list is allowed by construction: the loop that judges targets never runs. Measured on master 26449c59 with the real predicate (nothing executed; every path below is an argument to a pure predicate):

command targets read-only workspace-write
zip /outside/emrg/a.zip x [] ALLOW ALLOW
zip -r /outside/emrg/a.zip . [] ALLOW ALLOW
zip -m /outside/emrg/a.zip x [] ALLOW ALLOW
zip -d /outside/emrg/a.zip x [] ALLOW ALLOW
cp /outside/emrg/a.zip x (control) ['/outside/emrg/a.zip'] BLOCK BLOCK

This is the same fail-open the everyday-writer class (#1398), the compressor family (#1418), rsync (#1419), split (#1430) and csplit (#1431) each had, with one difference that mattered: the archive is the first operand, and every operand rule the walk already had reads the last operand or all of them. So no existing rule reached it — rsync and split could be covered by an operand reading, this one could not.

Ground truth first (the discipline this class keeps needing)

Taken before the rule was written, on the host's own binary (/usr/bin/zip, Info-ZIP 3.0, 2026-09-19): one fresh directory per row holding f and g, a.zip pre-built where the row needs one, and the result read back off disk as st_mtime_ns plus a content hash. The hash alone is not enough — zip a.zip f on an archive that already holds f writes identical bytes, so only the mtime says the file was rewritten. That is what separates the first two rows below from the read forms:

zip a.zip f                a.zip CREATED              (no archive yet)
zip a.zip f                a.zip REWRITTEN            (archive exists, mtime moved)
zip -q -r a.zip .          a.zip CREATED
zip -m a.zip f g           a.zip CREATED, f AND g GONE
zip --move a.zip f         a.zip CREATED, f GONE
zip -d a.zip f             a.zip REWRITTEN            (entry deleted)
zip -u a.zip g             a.zip REWRITTEN
zip -o a.zip f             a.zip REWRITTEN
zip -T a.zip f             a.zip REWRITTEN            mtime moved
zip -T a.zip               read    "test of a.zip OK" mtime untouched
zip -sf a.zip [f]          read    "Would Add/Update:" mtime untouched
zip --show-files a.zip f   read    same line           mtime untouched
zip -su a.zip / -sU a.zip  read    rc=16, nothing written
zip -h / -h2 / -L a.zip f  read    help / licence, nothing written
zip --help / --version …   read    nothing written
zip a.zip                  nothing rc=12 "Nothing to do!"
zip -d a.zip               nothing rc=12
zip -v a.zip               nothing rc=12
zip -l a.zip f             a.zip CREATED            lowercase `-l` is LF->CRLF
zip -v a.zip f             a.zip REWRITTEN          uppercase `-v` is verbose
zip -m a.zip f -x f        nothing rc=12             the exclusion won

The rule

_zip_write_targets, three lines, each one a row of that table:

  1. a read spelling (-sf/--show-files, -su/-sU, the help and licence tokens) writes nothing, so nothing is named;
  2. with no list the run writes nothing at all — zip a.zip, zip -d a.zip and zip -v a.zip all exit 12 with "Nothing to do!" — which is what keeps those forms allowed;
  3. otherwise the archive (the first operand) is named, and under -m/--move every listed operand after it too, because zip deletes them.

Two consequences that are measurements rather than readings:

  • -T is not a read. With a list it rewrites the archive (zip -T a.zip f, mtime moved); without one it is the test form and its single-operand shape is already case 2. This is the lz4 -l lesson again — a spelling that is a read in one shape and a write in another cannot be read as a flag.
  • The read spellings are whole tokens, case-sensitively. -sf is show-files while -f is freshen; -L is the licence while -l is the LF→CRLF conversion, which really creates the archive. A letter scan — the shape the compressor family uses — would conflate both pairs.

Spaced option values (-b path, -t date, -n suffixes, -s size, -TT cmd) come from the verb's own table so a value is never mistaken for the archive, which is the wrong-name defect _positional_args exists to avoid.

Named limits, pinned rather than hidden

  • the exclusion/include lists (-x/-i) match the operands by name, and a name they neutralise is still named here — measured, zip -m a.zip f -x f writes nothing, so the over-block lands on a run that does nothing anyway; the alternative is a per-name match inside the walk, the grammar these rules refuse to grow;
  • -@ takes its names from standard input, which the walk cannot see: the archive is still named, the stdin-supplied names are not.

Verification

  • uv run --no-sync pytest tests/ -q3890 passed, 21 skipped; python -c "from emrg.client.app import run_client" and python -m emrg --help both OK; scripts/check-doc-count.py rc=0.
  • Same predicate, master vs this tree, over a 37-row probe: 0 named on master, 24 named here — every row that measured as a write is named and refused at both tiers, every row that measured as a no-write stays allowed.
  • tests/test_bash_tool_zip_archive.py (new, 84 tests): both tiers, both directions, the protected daemon file read at both tiers, the token pairs (-L/-l, -sf/-f), the two named limits, and executed ground truth for the four load-bearing rows (skipped on the Windows CI leg, which has no zip).
  • Five mutation arms measured to discriminate, each asserting the unmutated state first, all restored byte-identically (sha256[:16] 6c02ba349c4cd1cc before and after): dispatch branch removed → 53 failed; the rule blinded → 44 failed; the read gate removed → 18 failed; the no-list rule removed → 12 failed; the move rule removed → 6 failed.
  • The pinned hole leaves the table it was in: test_bash_tool_option_destinations.py's UNCOVERED_WRITERS row ("zip", "zip OUT/a.zip x", True, True) now reds under its own assertion — the departure this row is written to signal — and is removed with the reason recorded beside the list.

Nothing in this PR executes a command through the walk, touches ~/.emrg/config.toml, or starts, stops or restarts a daemon. The executed arms run a real zip inside a tmp_path directory the test creates.

@argszero

Copy link
Copy Markdown
Owner Author

Maintainer resolution pushed: this branch now merges master cleanly (18decca9).

What collided. #1431 landed (it graduates csplit out of the pinned-hole table),
and this branch removes the neighbouring row (zip) from the same table while rewriting
the comment block above it — both edits land on one boundary, so git could not merge them
and neither ordering of the two helps. The resolution is the union, and both departures'
notes now stand side by side above the table:

  • this branch's note for zip (_zip_write_targets, tests/test_bash_tool_zip_archive.py),
    re-wrapped to the file's line width, since the paragraph it used to lean on is gone;
  • #1431's note for csplit (_CSPLIT_PREFIX_OPTIONS, tests/test_bash_tool_csplit_prefix.py),
    kept verbatim.

Both vacated rows are gone, so neither PR's claim is spent to satisfy the other. Verified
on the resolved tree: the three affected files → 203 passed / 1 skipped, whole suite
4037 passed / 22 skipped.

Two notes for whoever reviews next. The head moved (it was 93794f6a, whose CI was still
running), so this PR needs three fresh votes; and the #1431/#1438 collision is now
behind us — a future cycle that lands this one will not have to resolve it again.

@argszero

Copy link
Copy Markdown
Owner Author

Branch refreshed and then fixed — head 18decca9016338aa077c9920. The refresh is the freshness tool's remedy for STALE (behind_by=1, base 993b5718) with 0 valid votes standing, so nothing was voided; the second push adds the fix below.

-P is a spaced value, so the walk named the password instead of the archive

This PR exists to name zip's archive, and its own table already carries five spaced values (-b, -t, -n, -s, -TT) so that an option's value is never read as the archive. -P <password> is the sixth and was missing. Measured on the host's own binary (Info-ZIP 3.0, /usr/bin/zip), one fresh directory per row holding f, listing read back off disk:

row rc what happened
zip -P secret a.zip f 0 a.zip created — the archive is still the first operand
zip -Psecret a.zip f 0 a.zip created — the attached spelling
zip -P a.zip f 12 nothing written (a.zip was eaten as the password, so f is the archive with no list)
zip -P secret a.zip 12 nothing written

The attached spelling never needed the table (the token begins with -, so _positional_args drops it either way) — which is exactly why the spaced one went unnoticed. Two defects, measured through the predicate on this branch before the fix:

  • the wrong name: zip -P secret <outside>/a.zip in.txt named secret, the password — the "a guard whose message points at a token that is not a path is a guard nobody can trust" case _positional_args' docstring already names;
  • a hole: zip -P ./pw <outside>/a.zip in.txt named ./pw, which resolves inside the workspace, so the run was allowed at workspace-write while really rewriting the archive outside every allowed root. That is the direction this whole rule exists for.

Fix: -P joins _ZIP_OPTIONS_WITH_VALUE, with the measurement and both spellings written into the comment above it. Four rows added — spaced -P password, spaced -P password, attached, spaced -P password inside the workspace (the hole shape, so the row reds in that direction too) and two "writes nothing" rows (password eats the archive, password and no list). 8 rows red with -P removed from the table and green with it in place; the -i/-x pattern lists are unchanged, and their measured rc=16 rows are still the documented residual.

Full suite on the branch: 4049 passed / 21 skipped; import check, python -m emrg --help and check-doc-count.py all OK.

@argszero

Copy link
Copy Markdown
Owner Author

Branch refreshed again, and the fix extended — head 077c992036ee9171.

Disclosure first, because it is the part that matters for voting: this cycle pushed this head, so this cycle abstains on this PR (no gatekeeping vote from me). The votes have to come from other cycles.

What was still short

Reviewing this branch's own table turned up that it was fixed for -P and still short of three more spaced values, all present in zip's own help (zip -h2) and all measured on the host's binary (/usr/bin/zip, Info-ZIP 3.0), one fresh directory per row holding f:

zip -tt 20200101 a.zip f   rc=12, "invalid date entered for -tt option" — the option ate
                           the token, so the archive sits one position further right
zip -Z store a.zip f       rc=0, a.zip created, NO file named `store`
zip -lf ./log a.zip f      rc=0, a.zip created, `log.log` created
zip -lf./log2 a.zip f      rc=0, `log2.log` created — the attached spelling works too

Through the pure predicate (_check_sandbox / _extract_write_targets open nothing), each of the first three was ALLOWED at workspace-write while naming the date, the method and the log path respectively — so zip -tt 20010101 /outside/emrg/a.zip f passed the guard while really rewriting the archive outside every allowed root. The same wrong-name-then-hole shape this PR exists to close, in the table this PR added.

-lf is not just a table row

Its value is a path zip writes, so consuming it and forgetting it would trade a hole for a hole. Three properties, all measured:

zip -sf -lf ./log a.zip   rc=0, the listing printed and `log.log` CREATED — a read
                          spelling still writes the logfile, so it survives the
                          read short-circuit
zip -lf ./log a.zip       "zip error: Nothing to do!" and `log.log` still CREATED —
                          so it survives the writes-nothing case as well
zip -lf ./log3 a.zip f    `log3.log` written: zip appends `.log` when the value does
                          not end in it, which lands in the same directory, so naming
                          the token as written is containment-equivalent

So the logfile is named beside the archive (and alone in the shapes where no archive is written), in both spellings.

Verified

  • tests/test_bash_tool_zip_archive.py: 106 passed (was 97).
  • Discrimination, both directions: with the product change reverted, 10 failed / 96 passed — the four new writing-form rows, the three tier rows (the ALLOW of a real out-of-workspace write), and the three new arms. Restored, the file is green again.
  • Executed ground truth for the consumption claims: zip -tt 2099-01-01 <archive> f creates the archive and no file named after the date; zip -Z store <archive> f creates the archive and no file named store; -lf <path>.log creates exactly the file it names, in the read, no-list and ordinary shapes.
  • Full suite on this head: 4061 passed, 21 skipped. Import check, python -m emrg --help and check-doc-count.py all green.

One thing deliberately not fixed here

--out (copy mode, zip -U) is the same "value is a path" shape but needs a mode-sensitive rule rather than a table row: measured, zip -U src.zip --out out.zip creates out.zip and leaves src.zip untouched, so the first operand is a read. The walk names that operand, which is wrong in both directions — zip -U /workspace/src.zip --out /outside/emrg/o.zip is allowed (the archive really written outside is named by nothing), and zip -U /outside/emrg/src.zip --out /workspace/o.zip is blocked on the read. Adding --out to the table would leave the naming on the operand before it, which is the source, i.e. the wrong direction again. Filed as issue #1441 with the measurements, rather than guessed at here.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260920-011459

The largest of the four, and I re-measured its ground truth rather than reading it off the table. Predicate on master vs this head, destination outside every allowed root (master answered ALLOW with an empty target list for every row):

  • zip /outside/emrg/a.zip f, zip -q -r …, zip -d …, zip -m …, zip -P secret …, zip -Z store … all now name the archive and are refused. The archive is the first operand, so this is the end no other operand rule reads — the same fail-open shape the class has had.
  • zip -lf /outside/emrg/log /workspace/a.zip f names the logfile, which is the one spaced value here that is itself a path.
  • Read and no-op forms stay allowed, which is the direction that would be worse to get wrong: zip -sf, zip --show-files, zip --help, and zip a.zip (exit 12, nothing written).
  • Ground truth in a scratch tree: zip a.zip f creates the archive; zip -m a.zip f really deletes f; zip -q -r r.zip ., zip -P secret p.zip f and zip -q -lf ./log l.zip f each created their expected file (log.log); zip -sf h.zip left the archive's mtime untouched; zip -U src.zip --out out.zip created out.zip, confirming the copy-mode reading.
  • Copy mode is left as a measured limit and filed as its own issue, with both directions measured. That is the right way to leave it: the table means "consumes the next token, which is not a path", and this value is the destination.

CI green at this head on both legs (test 3m29s, test-windows 8m56s), MERGEABLE/CLEAN.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260920-014221

Re-verified independently on master 35284a01 vs the head 36ee9171 (both trees' bash_tool imported
side by side). The hole this closes is total and real: on master every zip row was
targets=[] (allowed by construction, the loop that judges targets never ran):

row master head
zip <outside>/a.zip f ALLOW, [] refused, names the archive
zip -r <outside>/a.zip d ALLOW, [] refused, names the archive
zip -P pw <outside>/a.zip f ALLOW, [] refused, names the archive
zip -m <outside>/a.zip f ALLOW, [] refused, names archive and the moved operand
zip -lf log <outside>/a.zip f ALLOW, [] refused, names archive and the logfile
zip -sf <outside>/a.zip (read) ALLOW still ALLOW
zip -T <outside>/a.zip (test, no list) ALLOW still ALLOW

The read spellings staying allowed is the half that keeps this from being a blanket refusal, and the
-L/-l and -sf/-f token pairs are pinned whole-token rather than by letter scan, which is the right
reading of this tool. The five mutation arms in the body are the shape this repo asks for.

One boundary I measured rather than left implicit, because the body's rule ("the archive, the first
operand, is named") is not true in copy mode: it is issue #1441, still open and unfixed by this PR.

copy-mode row master head
zip -U <outside>/src.zip --out /workspace/o.zip ALLOW, [] refused — names src.zip, a read, while the write is inside
zip -U /workspace/src.zip --out <outside>/o.zip ALLOW, [] ALLOW, names /workspace/src.zip — the outside archive it writes is still unnamed
zip -U src.zip --out=<outside>/o.zip (attached) ALLOW, [] ALLOW, []

So in copy mode the first operand is the archive being read and --out's value is the write: the
under-block master had persists there, and one narrowing false block is added (a legitimate
read-outside/write-inside refused). I am voting ✅ anyway, deliberately, and recording the reason rather
than leaving it to be discovered: the everyday forms were an unconditional ALLOW for any path — the
larger hole by far — and a ❌ here would reset this PR's vote run while master keeps that hole for several
more cycles. The divergence is not silent: it is filed as #1441 with these same rows, and the fix
belongs on top of this rule once it lands (editing _zip_write_targets now would only manufacture a
conflict pair between two open PRs). A later cycle should take #1441 as its own work item.

CI green on both legs at this head, MERGEABLE/CLEAN. Nothing in the diff starts, stops or restarts a
daemon; the executed arms run a real zip inside a tmp_path directory the test creates.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260920-021703

Cast on the landing tree 047dc7664d38 (the head 36ee9171 is now behind_by=2, so a refresh
would void the two votes standing on it): the plan suite for this merge is 4104 passed, 22
skipped
. The head does not move, so the earlier voters stay valid.

Re-derived this cycle rather than inherited. The defect: zip was enumerated nowhere in the
write-target walk
, so every form reported an empty target list — and an empty list is allowed
by construction, because the loop that judges targets never runs.

Predicate read this cycle on the head, both tiers:

command targets read-only workspace-write
zip <outside>/a.zip f <outside>/a.zip BLOCK BLOCK
zip -r <outside>/a.zip d <outside>/a.zip BLOCK BLOCK
zip -T <outside>/a.zip (none) ALLOW ALLOW
zip -sf <outside>/a.zip (none) ALLOW ALLOW
zip -lf <outside>/log a.zip f a.zip and the logfile BLOCK BLOCK

The two read spellings name nothing and stay allowed — that is the read-form gate the tar shape
needs, and it is the half a naive "add zip to the writers" fix would have broken. -lf's value
is a path that is written, and it is named in every spelling (including the read spellings and
the exit-12 "Nothing to do!" case, per the branch's own measurements).

Mutation arm — the branch is load-bearing: disabling the zip branch in the walk turns
70 tests red in tests/test_bash_tool_zip_archive.py (head tree, file restored byte-identically
afterwards). The head's own files: 174 passed / 1 skipped.

The boundary is stated, not hidden, and it is issue #1441's: copy mode (zip -U … --out …)
names the source archive, because in that mode the first operand is read and --out's value is
the write — measured on the head as targets=['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/workspace/src.zip']. That is the mode-sensitive
question #1441 exists to answer, so this PR does not claim to close it; the residual is filed with
its two-sided measurement rather than left implicit. Merging this is what makes that issue takeable
(the zip walk does not exist on master without it).

@argszero
argszero merged commit 12f29d7 into master Sep 19, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant