Skip to content

emrg: a bare dash is a stream, not a path to the compressors - #1440

Merged
argszero merged 1 commit into
masterfrom
fix/a-bare-dash-is-a-stream-not-a-path
Sep 19, 2026
Merged

argszero merged 1 commit into
masterfrom
fix/a-bare-dash-is-a-stream-not-a-path

Conversation

@argszero

Copy link
Copy Markdown
Owner

The defect

A bare - operand is a stream, not a path, for the compressor family — and the
write-target walk named it, so read-only refused commands that write nothing:

gzip -      -> targets=['-']  read-only: blocked destructive write targeting '-'
zstd -      -> targets=['-']  read-only: blocked destructive write targeting '-'
lz4 -       -> targets=['-']  read-only: blocked destructive write targeting '-'

That is a false block, the direction the walk's own record treats as the costlier
one: an empty target list is a hole that can be argued out of, a refusal is a command
a reader cannot run. gzip - is the everyday "compress stdin to stdout" idiom.

The measurement (host, 2026-09-19, one fresh directory per row, listing read back off disk)

row rc file created
gzip - xz - bzip2 - zstd - compress - lz4 - 0 none
gzip -9 - gzip -- - xz -9 - bzip2 -9 - zstd -19 - lz4 -9 - 0 none
gzip -d - 1 (unexpected end of file) none
lz4 - - lz4 -t - lz4 f - 0 / 44 / 0 none (lz4 f - names stdout, so nothing lands beside f)
gzip - f gzip f - xz - f zstd - f 0 f IS compressed
touch - truncate -s0 - mv src.txt - cp src.txt - 0 the file named - IS created

The fix

_without_the_stream_operand drops the token one operand at a time:

  • the compressor family applies it to the operands of a write form;
  • lz4 applies it in the multi-input branch, and its last-operand branch answers with
    nothing when the destination itself is - (lz4 f - writes nothing beside f).

Two things the measurement decided against:

  • Not a run-level answer in _compressor_operand_is_a_read. That gate answers once
    per run, so teaching it the bare - would have made gzip - f name nothing at all
    while gzip really writes f.gz and removes f — a hole in exchange for the false
    block. The dash beside a file row pins this (['f'], still refused).
  • Not in _positional_args. There every verb would inherit the drop, and to the
    everyday writers a bare - is a path: touch - / truncate -s0 - / mv src.txt -
    / cp src.txt - really create the file named - in the same geometry.
    test_the_drop_is_the_compressors_alone is the boundary row that reds if it moves.

The drop lives in the branch, not in a name list, so the twins added by #1439
(zstdmt, unlz4, lz4c) inherit it rather than needing a second fix.

Verification

  • 17 new rows red on pristine master (f1db…-equivalent tree edba48ca) and green here,
    including three arm tests that flip the drop and the per-operand rule.
  • uv run pytest tests/ — 3976 passed, 21 skipped.
  • uv run python -c "from emrg.client.app import run_client" and uv run python -m emrg --help — OK.
  • scripts/check-doc-count.py — OK (no tracked file states the Python count).

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260920-004551

Reviewed the diff and re-measured every load-bearing claim myself rather than reading it out of the PR text.

What the patch does: _option_destination_values already dropped a bare -, but the compressor path reached _extract_write_targets through a different route, so a lone - (stdin/stdout) was read as a path operand.

Independently verified, both directions:

  • On master edba48ca — the false block is real: gzip -targets=['-']; likewise gzip -9 -, gzip -- -, bzip2 -, xz -.
  • On the head aa34fa4b — all of those → targets=[]. The false block is gone.
  • Crucially, no hole is opened: gzip - /outside/emrg/f is still blocked on the head. The drop is per-operand, so the real path is still named and still refused.
  • The boundary rows really do create a file literally named - on disk (measured, not assumed), which is what makes them a meaningful pair.

No defect found. Both CI legs green at this head (test 3m49s, test-windows 9m12s), merge state MERGEABLE/CLEAN.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260920-011459

Re-verified on this head independently (predicate probes only, nothing executed, destination outside every allowed root):

  • gzip - → master names ['-']; this head names [] — the false block is gone.
  • gzip - /outside/emrg/f → still names ['/outside/emrg/f'] and is refused at both tiers, so no hole was opened in the direction the rule exists for.
  • Ground truth from a scratch tree: gzip - really reads stdin and writes stdout — a file literally named - in the cwd kept its own content afterwards — while gzip - real2.txt really created real2.txt.gz. So - is a stream, and a second operand is a real write.

CI green at this head on both legs (test 3m49s, test-windows 9m12s), MERGEABLE/CLEAN.

@how2how2how2-arch

Copy link
Copy Markdown
Contributor

Independent verification of aa34fa4 — and the merge interaction with #1439

Verified on the head aa34fa4b, staged from the git objects into a pristine directory with no .git, so nothing here runs against a working checkout.

Suite arms

tree result
master edba48c 17 failed, 3934 passed, 26 skipped
this head aa34fa4b 17 failed, 3954 passed, 26 skipped
this head's two test files dropped onto unpatched master 17 failed, 122 passed

The 17 failures are the known .git-dependent staging artefacts, and the failure set is byte-identical to master's — no regression. The ablation arm is the one that matters: copying only your two test files onto unpatched master flips every dash row back to red, so the rows ride on the drop and not on something else in the walk.

Predicate and verdict, master vs this head (predicate only, workdir="/workspace", read-only tier)

command master edba48c this head
gzip - ['-'] → BLOCK [] → ALLOW
gzip -- - ['-'] → BLOCK [] → ALLOW
lz4 - ['-'] → BLOCK [] → ALLOW
lz4 f - ['-'] → BLOCK [] → ALLOW
gzip - f ['-', 'f'] → BLOCK ['f'] → BLOCK
lz4 -m f - ['f', '-'] → BLOCK ['f'] → BLOCK

So the premise of the change is exact: on master the bare dash is named for this family, and the fix spares the token while keeping the file beside it.

Ground truth, re-measured on this host (fresh directory per row, listing read back off disk)

Creates nothing, rc=0: gzip -, gzip -9 -, gzip -- -, xz -, bzip2 -, zstd -, compress -, lz4 -, lz4 -9 -, lz4 f -, zstdmt -, unlz4 -, lz4c -. gzip -d - is rc=1 (unexpected end of file), nothing created — as your docstring states.

Writes, in a clean directory holding only f: gzip - f and gzip f - produce f.gz (and remove f); xz - f produces f.xz; lz4 -m f - derives f.lz4; zstd - f derives f.zst; zstdmt f derives f.zst; lz4c f derives f.lz4. Every positive row holds under re-measurement.

Both halves of the boundary you drew are confirmed: touch -, truncate -s0 -, mv src.txt - and cp src.txt - each create the file named - (rc=0), while rm - looks one up (rc=1). Keeping the drop out of _positional_args is the measured call.

An adjacent positive control for that same scope decision, for whoever reviews the sibling PR: to zip the bare dash is a stream too — zip - is rc=12 Nothing to do! (-), and zip - f is rc=0 that adds f to stdout (nothing appears on disk). Zip already names nothing there, so the family-scoped drop leaves it exactly as it was.

Merge interaction with #1439

git merge-tree --write-tree r1439 r1440 auto-merges emrg/tools/bash_tool.py and conflicts only in the two test files (both sides append rows and arms). I resolved those additively by hand and ran the full suite on the merged tree: 17 failed, 3974 passed — same 17, both PRs' arms green together.

On that merged tree the drop reaches #1439's new names with no second edit needed:

zstdmt -   []            ALLOW      (zstdmt lands in _COMPRESSOR_VERBS)
unlz4 -    []            ALLOW      (lz4 branch)
lz4c -     []            ALLOW
zstdmt /tmp/x/f -   ['/tmp/x/f']   BLOCK   (the file is still named, the token is not)

So #1440 and #1439 compose in the dash dimension — the reason to prefer landing this one first is on #1439, where it is stated as a measurement.

One minor note

The docstring's "same bytes" claim re-verifies today, but the pinned digests no longer match this host's binaries: /opt/homebrew/bin/zstdmt and Cellar/zstd/1.5.7_1/bin/zstd are both aff8169fb421…, and unlz4 / lz4c / lz4cat / Cellar/lz4/1.10.0/bin/lz4 are all b7dccdc84a76… — i.e. one file each, but not the sums recorded in the docstring (the zstd cellar path moved to 1.5.7_1). The identity claim stands; the digest literal is the part that will rot. Not a blocker for this PR.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260920-014221

Independently re-verified on the current master (edba48ca) vs the head aa34fa4b, by importing both
trees' bash_tool side by side (predicate only, nothing executed for the verdict rows):

command master head
gzip - / zstd - / lz4 - / gzip -9 - / gzip -- - / gzip -d - targets=['-'], read-only refuses targets=[], read-only allows

That is the false block the PR names: - is the documented stream operand and these runs write no file,
so refusing them at the tier whose documented flow is to keep working is the costly direction.

Ground truth re-measured here by execution, both directions (fresh directory, listing read back off
disk): gzip - < f → rc=0, no file created; touch - → rc=0 and the file named - really
exists
, so naming it there stays correct. The controls confirm the drop is scoped rather than a blanket
"ignore a bare dash": gzip - f still names f (real run: rc=0, f.gz created, f gone) and
touch - still names -; gzip -c - was already targets=[] on master and is unchanged.

The mutation arm was run rather than assumed: with _without_the_stream_operand replaced by a pass-through
on the head tree, the two compressor test files go 13 failed / 126 passed (including
test_the_stream_operand_is_what_spares_the_bare_dash), against 139 passed unmutated — so the rows depend
on the fix they pin.

CI green on both legs at this head (test 3m49s, test-windows 9m12s), MERGEABLE/CLEAN. No test in
the diff starts, stops or restarts a daemon.

@argszero
argszero merged commit 35284a0 into master Sep 19, 2026
2 checks passed
argszero pushed a commit that referenced this pull request Sep 19, 2026
…nded)

Both sides are additive rows in the same tuple: this branch's twin-name rows
(zstdmt/lz4c/unlz4/lz4cat) and master's bare-dash rows. Keep both sets; no row
from either side is dropped or reworded.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants