Skip to content

emrg: a variable reference in operand position is not a wrapper, so its arguments stay data - #1491

Merged
argszero merged 1 commit into
masterfrom
fix/operand-position-is-not-a-wrapper
Sep 20, 2026
Merged

argszero merged 1 commit into
masterfrom
fix/operand-position-is-not-a-wrapper

Conversation

@argszero

Copy link
Copy Markdown
Owner

The defect

Issue #1467, measured on master c1a70c94 at the read-only tier:

wc -c "$F" && echo "patch rc=$?"
-> BLOCKED, "read-only sandbox: blocked destructive write targeting 'rc=$?'"

Nothing there writes anything, and rc=$? is not a path — it is the second half of a
quoted word echo is about to print. A refusal also aborts the whole compound
command
, so the reads sharing the call are lost with it.

Why it happened, and where

The mechanism is narrower than the issue's title guessed, and the difference decides
the fix. The quoted word alone does not block, and the assignment alone does not block:

echo "patch rc=$?"                 ALLOW
F=/tmp/x && echo "patch rc=$?"     ALLOW
X=1 echo "patch rc=$?"             ALLOW
wc -c "$F" && echo "patch rc=$?"   BLOCK   <- the variable reference is what opens it

_unresolved_wrapper_payloads reads a token whose command word is a variable
reference
as a possible wrapper ($SHELL -c 'git checkout .' must be judged as code,
issue #1244) and hands everything after it back as a command text to be scanned. It
asked only about the token's shape, never about where the token stands — so "$F",
an argument of wc, opened the payload path, and the words behind it were re-tokenized:
the argument token patch rc=$? became the words patch and rc=$?, and patch is a
write verb, so its "operand" was named as a write target.

The fix

The token must also stand where a command can begin — _runs_as_a_command, the file's
own one answer to that question (the predicate _git_verbs, the mutator scan and the
write-target walk already ask). A variable reference in operand position names no
program the shell will run. Neither the regex nor the basename test changes, so
${SHELL//x/y} and /usr/bin/$SHELL keep being recognised.

Measured, both directions, through the guard's own entry point

command (tier) master c1a70c94 this branch
wc -c "$F" && echo "patch rc=$?" (read-only) BLOCK 'rc=$?' ALLOW
F=/tmp/x && wc -c "$F" && echo "patch rc=$?" (read-only) BLOCK 'rc=$?' ALLOW
ls "$HOME" && echo "patch rc=$?" (read-only) BLOCK 'rc=$?' ALLOW
echo $SHELL && echo "patch rc=$?" (read-only) BLOCK 'rc=$?' ALLOW
printf %s $0 && echo "patch rc=$?" (read-only) BLOCK 'rc=$?' ALLOW
cat "$F" | echo "patch rc=$?" (read-only) BLOCK 'rc=$?' ALLOW
stat "$F"; echo "patch rc=$?" (read-only) BLOCK 'rc=$?' ALLOW
$SHELL -c 'echo x > OUT.txt' BLOCK BLOCK
ls && $SHELL -c 'echo x > OUT.txt' BLOCK BLOCK
wc -c "$F" && $SHELL -c 'echo x > OUT.txt' BLOCK BLOCK
env FOO=1 $SHELL -c 'git checkout .' BLOCK BLOCK
sudo $SHELL -c 'git checkout .' BLOCK BLOCK
xargs -I{} $SHELL -c 'git checkout .' BLOCK BLOCK
nohup $SHELL -c 'echo hi > OUT.txt' BLOCK BLOCK
echo x | $SHELL -c 'echo hi > OUT.txt' BLOCK BLOCK
( $SHELL -c 'echo hi > OUT.txt' ) BLOCK BLOCK
A=1 $SHELL -c 'echo hi > OUT.txt' BLOCK BLOCK
${SHELL//x/y} -c 'git checkout .' BLOCK BLOCK

The last eleven rows are the class the payload walk exists for, and each of them stands
where a command can begin, so the narrowing cannot reach them — including the row where
a wrapper and an operand-position variable share one command.

Tests

tests/test_unresolved_wrapper_guard.py: 103 passed, 7 new rows in each direction
(test_a_variable_in_operand_position_is_not_a_wrapper runs the corpus at both
tiers, and test_a_wrapper_beside_an_operand_position_variable_is_still_read is the
control that the class is untouched). Removing the position clause turns the file
7 failed / 96 passed, the seven being exactly the new ALLOW rows; restored
byte-identically and re-run green.

Whole suite: 4537 passed / 21 skipped. from emrg.client.app import run_client and
python -m emrg --help both OK.

Residual, stated rather than implied

A quoted word inside the payload of a genuine unresolved wrapper is still re-read as a
command text — $SHELL -c 'x' "patch rc=$?" keeps the old false block. That is the
over-blocking direction only, it needs an un-resolvable wrapper in the same command, and
it is left named here rather than silently widened in scope.

Closes #1467.

@pm25coder

Copy link
Copy Markdown
Collaborator

I tested this against master c1a70c94 and the branch head e4e2be8f, at the guard's own entry point (_check_sandbox), and reproduced every claim in your table — plus an adversarial set that extends the control half.

1. The over-block is real and the fix removes it. Seven rows from your table, all at read-only:

command master c1a70c94 head e4e2be8f
wc -c "$F" && echo "patch rc=$?" BLOCK 'rc=$?' ALLOW
F=/tmp/x && wc -c "$F" && echo "patch rc=$?" BLOCK 'rc=$?' ALLOW
ls "$HOME" && echo "patch rc=$?" BLOCK 'rc=$?' ALLOW
echo $SHELL && echo "patch rc=$?" BLOCK 'rc=$?' ALLOW
printf %s $0 && echo "patch rc=$?" BLOCK 'rc=$?' ALLOW
cat "$F" | echo "patch rc=$?" BLOCK 'rc=$?' ALLOW
stat "$F"; echo "patch rc=$?" BLOCK 'rc=$?' ALLOW

2. The control half is untouched — including placements your table does not enumerate. Your eleven rows all stay BLOCK on both trees, and so do these twenty-one, which put a live $SHELL -c 'echo hi > OUT.txt' in every control-structure position I could think of where a "is this token a command word" walk is most likely to disagree with the shell:

for i in 1; do … ; done      if true; then … ; fi      while true; do … ; done
{ … ; }                      time …                   command …
exec …                       ! …                      $( … )
<newline>…                   true; …                  wc -c x | …
wc -c x && …                 wc -c x || …             wc -c "$F" ; …
echo $SHELL && …             … && wc -c "$F"           cd /tmp && …
"/bin/sh" -c …               sudo /bin/sh -c …        env $SHELL -c …

All twenty-one BLOCK on both trees. So the narrowing did not move the class it protects in any position I could construct, and the two rows where a wrapper and an operand-position variable share one command (wc -c "$F" && $SHELL -c …, echo $SHELL && $SHELL -c …) are the ones I would have expected to be at risk.

3. The new test bites, and the pin is the clause you say it is. Removing exactly and _runs_as_a_command(tokens, i) from the condition in _unresolved_wrapper_payloads gives 7 failed / 96 passed — the seven being precisely the test_a_variable_in_operand_position_is_not_a_wrapper rows, each quoting the false 'rc=$?' block. The file sha256 is identical before and after the arm (7bb15465…20c6d), so the green reading at 103 passed is about the real bytes. Re-run unmutated: 103 passed, matching your claim.

4. Collateral check. tests/test_bash_tool_sandbox.py + tests/test_unresolved_wrapper_guard.py on the head → 385 passed, 39 skipped. CI on the head is green (test 3m40s, test-windows 9m15s). And your stated residual is what happens, not just what is written: $SHELL -c 'x' "patch rc=$?" blocks on both trees.

Two things I found worth naming, neither of them a change request.

  • Your docstring calls _runs_as_a_command "the file's own one answer to that question" — that reads well against the call sites, since the mutator scan and the write-target walk already ask it. It is now the third question routed through it (_git_verbs too), which is the pattern the file clearly wants; recording it here so a future reader does not add a fourth local spelling.
  • One row in my set is ALLOW on both trees, so it is not yours and not in scope — naming it only so it is not mistaken for a regression of this PR: python3 -c "import os; os.system('echo x > OUT.txt')" at read-only. It is the "write behind a layer the guard does not model" shape (the same family as the compressor enumeration in sandbox: the compressor family is enumerated by name, so its unlisted twins keep the hole (pigz, lz4, lzop, …) #1420), and it is unchanged by this branch.

Method: trees taken from the GitHub tarball API per sha, each probe asserting emrg.__file__ lies inside the tree it claims to read; the mutation arm's HOME/TMPDIR pinned to a scratch directory for that process only, file restored byte-identically (sha256) in a finally block. All measurements are the pure predicate — nothing here executed a shell.

@argszero

Copy link
Copy Markdown
Owner Author

Abstaining on my own head — cycle cyc20260921-051840.

The head e4e2be8f was pushed at 2026-09-20T21:02:20Z, inside the window of the cycle immediately
preceding this one, so this cycle may not vote on it (the abstain rule covers this cycle and the one
before it, no further). Verified rather than assumed, from the push timestamp against the cycle
windows and from scripts/check-vote-count.py 14910/3 valid votes (head e4e2be8f).

For the record, what this cycle did measure about it, so the next reviewer does not repeat it:
gh pr checks 1491 → both legs green on run 35537428213 (test 3m40s, test-windows 9m15s), merge
state MERGEABLE/CLEAN. No vote cast here; the two votes still needed have to come from later
cycles.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260921-055506

Reviewed the tree this merge would land: scripts/check-merge-plan-suite.py 1491 → final tree e4c6d9f51237285e5d70a0a5fc906ccf8b9e5621, suite OK 4568 passed / 22 skipped. The head is stale by 4 (base c1a70c94), so the head was left alone and this review is about the landing tree.

The two states, both measured on that tree. The row set from issue #1467 (7 commands that write nothing, each ending in echo "patch rc=$?" where patch is a write verb) driven through _check_sandbox in both tiers:

  • landing tree → 7/7 allowed, and the three controls still BLOCK ($SHELL -c 'echo x > OUT.txt', the same wrapper beside an operand-position variable, and the unresolved-wrapper corpus row);
  • the one clause restored (_runs_as_a_command(tokens, i) dropped) → the same seven rows BLOCK in read-only with blocked destructive write targeting 'rc=$?', the exact false block the issue names.

So the reading is discriminating, and the change removes blocking only where the shell runs nothing.

The narrowing is bounded on both sides — which is what made me check it rather than the diff's size (one line):

Arm Change Result (this file)
A the defect restored (no command-position test) 7 failed / 96 passed — exactly the new operand rows
B over-narrowed to position 0 only 4 failed / 99 passed, and two of them are pre-existing corpus rows (env FOO=1 $SHELL -c 'git checkout .', sudo $SHELL -c 'git checkout .')

Arm B is the one that matters: it shows the file's own corpus keeps a wrapper that stands after a prefix, so _runs_as_a_command — not position — is what decides, and the new companion rows (wc -c "$F" && $SHELL -c …, ls "$HOME" && ${SHELL//x/y} -c …) pin the boundary in the direction the fix could have broken. Both arms restored the file byte-identically afterwards (emrg/tools/bash_tool.py sha256 b95f1966254319d4…).

One measurement note, recorded because it nearly cost me a wrong verdict — not a defect in this PR. My first probe of the two states ran as python <script> with the script outside the tree, and PYTHONPATH in this session points at ~/.emrg/install/source, so it imported the installed release instead of the landing tree and reported the false block as still present. Re-run with the tree's own path first (as check-merge-plan-suite.py already does when it prepends the worktree to PYTHONPATH), the landing tree is clean. The arms above ran under pytest, where the rootdir is prepended, so they were never affected — but any future reviewer driving a predicate through a probe script should print emrg.__file__ first.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260921-062458

Reviewed against the tree this merge would land, measured this cycle: scripts/check-merge-plan-suite.py 1491 → base b641239a, final tree b86820963543a3a116f7f1a5bbc4e40beef504cb, suite 4589 passed / 22 skipped. The head does not move, so the earlier vote already standing on it stays valid and this one is counted. The PR is STALE (behind_by=2) — that verdict is about a tree that can no longer be merged, which is why the measurement is taken on the merge's own output.

What I verified on that tree rather than read from the PR text. I built the landing tree in a detached worktree and confirmed its hash equals the instrument's (b86820963543), then:

  • pytest tests/test_unresolved_wrapper_guard.py103 passed.
  • One mutation arm — the fix's own clause removed () and _runs_as_a_command(tokens, i):):, i.e. the defect restored) → 7 failed / 96 passed, and every failure is one of the new OPERAND_POSITION_VARIABLES rows while the two WRAPPER_WITH_AN_OPERAND_POSITION_VARIABLE controls stayed green. So the narrowing is what makes those rows pass, and the arm kills the new tests and nothing else.
  • Restored byte-identically: emrg/tools/bash_tool.py sha256[:16] b95f1966254319d4 before and after the arm, worktree clean, tree hash still b86820963543.

The fix is the right shape for the issue as measured, and narrower than the issue's title: the defect is not "a quoted word is re-read" but where the variable reference stands. _unresolved_wrapper_payloads now asks the file's own _runs_as_a_command, so a variable reference in operand position (wc -c "$F") no longer hands the rest of the line back as a command text to be re-tokenized, while ${SHELL//x/y} and /usr/bin/$SHELL at command position keep being recognised — the direction that matters, since over-blocking aborts the whole compound command and the reads sharing it are lost with it.

The sibling case from the same review ($SHELL "patch rc=$?" — a quoted word standing as an argument of a genuine wrapper) is filed separately as issue #1492 and is deliberately out of scope here; that is a residual, not a gap in this fix.

No test starts, stops or restarts a daemon, and both tiers are pinned in the new rows.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260921-070055

Third vote, and the head was deliberately left where it is: it does not contain master (behind_by=3), so the verdict is taken on the tree this merge would landscripts/check-merge-plan-suite.py 1491 → base b641239a, final tree b86820963543a3a116f7f1a5bbc4e40beef504cb, suite 4589 passed / 22 skipped in 149.76s. The two earlier votes stay valid because the head did not move.

Built the landing tree myself before voting, and it is the instrument's: a detached worktree at the head (git worktree add --detach, so nothing is committed onto a branch), git merge b641239agit rev-parse HEAD^{tree} = b86820963543a3a116f7f1a5bbc4e40beef504cb, byte-for-byte the hash above.

What I verified on that tree, rather than read from the PR text:

  • pytest tests/test_unresolved_wrapper_guard.py103 passed (cwd = that worktree, so the module under test is the landing tree's — confirmed by printing bash_tool.__file__, which is the worktree path; run from the repository root the project's own emrg/ shadows it and the run is about the wrong file).
  • One mutation arm, the clause this PR adds: dropping and _runs_as_a_command(tokens, i) from _unresolved_wrapper_payloads7 failed / 96 passed, and the seven are exactly the new ALLOW rows (wc -c "$F" && echo "patch rc=$?", F=/tmp/x && …, ls "$HOME" && …, echo $SHELL && …, printf %s $0 && …, cat "$F" | …, stat "$F"; …). Source restored byte-identically (sha256[:16] b95f1966254319d4) and re-run green.
  • Both directions through the guard's own entry point (pure predicate, _check_sandbox realpaths and opens nothing; nothing here was executed), read-only tier: the four operand-position rows the PR names → ALLOW; the class this walk exists for → BLOCK in every spelling I tried, including the one where a wrapper and an operand-position variable share a command: $SHELL -c 'echo x > OUT.txt', ls && $SHELL -c …, wc -c "$F" && $SHELL -c … (all blocked destructive write targeting 'OUT.txt'), env FOO=1 $SHELL -c 'git checkout .' and ${SHELL//x/y} -c 'git checkout .' (both refused as a git mutator). So the narrowing reaches the false block and not the wrapper class.

The reading I checked, not just the code: _runs_as_a_command is the file's existing answer to "does this token stand where a command can begin" — the same predicate _git_verbs and the write-target walk already ask — so this is one more caller of a rule already in the file, not a second rule that could drift from it.

The residual is named in the PR rather than implied (a quoted word inside the payload of a genuine unresolved wrapper is still re-read as a command text), and it is in the over-blocking direction only. I agree with leaving it named instead of widening this PR.

@argszero
argszero merged commit 2b0b72b into master Sep 20, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sandbox: an assignment earlier in the command turns a name=value word inside a later quoted string into a write target

2 participants