emrg: a Windows path is absolute for the shell that will run it (#1261)

[argszero]

Fixes issue #1261 — the second of the two tokenizer defects found while landing #1260.

What was wrong

The guard tokenises with shlex in POSIX mode, where a backslash is an escape; a Windows shell (cmd.exe, the bash tool's subprocess shell on that platform) treats the same character as a path separator. A Windows spelling therefore reached every rule with its separators deleted:

_split_command_tokens(r'echo x > C:\Users\x\out.txt')
  -> ['echo', 'x', '>', 'C:Usersxout.txt']
_is_absolute_path('C:Usersxout.txt')  ->  False      # not absolute
_check_sandbox(..., 'workspace-write') ->  True      # "relative, so in-workspace"

At workspace-write an absolute path outside the workspace was read as a relative name and allowed, while the same write spelled /Users/x/out.txt was refused. read-only was unaffected — that tier refuses every target without asking where it resolves.

Measured on master cca0b8dc, the corpus below (not a hand-picked example): 10 of 17 Windows spellings wrong, every one an escape — redirect, append, rm -rf, rm -rf of a drive root, mv, cp (with and without a trailing separator), sed -i, find -delete, and the same redirect behind a && chain. \\server\share\… was already blocked, but by accident: the mangled token kept its leading backslash.

The fix

On a Windows shell, substitute a placeholder for each backslash in the copy the guard tokenises — never in the command that is executed — and restore it in the tokens on the way out. Both tokenizers go through it (_split_command_tokens for write targets and cd operands, _tokenize_command for git verbs and wrapper payloads), including their unparseable-input fallbacks.

A character substitution rather than a "mangled form → raw spelling" lookup, deliberately: a lookup has a silent failure mode — a key that fails to match leaves the path mangled and every rule reading it inert — while a character that was never removed cannot fail to be restored. With the spelling intact, a drive-rooted path reads as absolute wherever the guard runs, and containment canonicalises the separator so the answer does not depend on which os.path the guard itself is running under.

Measured on the corpus, in BOTH platform states

The platform axis is a module constant (_WINDOWS_SHELL), so the Windows arm is exercisable anywhere instead of only where the host happens to be. Forced Windows vs forced POSIX, same corpus, same harness:

                                          master   fixed (forced WINDOWS)   fixed (forced POSIX)
outside the workspace (12 spellings)        ALLOW       BLOCK                     ALLOW
inside the workspace (3 spellings)          ALLOW       ALLOW                     ALLOW
POSIX corpus (7 spellings)                expected    expected                  expected
wrong verdicts                             10/17       0/17                      10/17

The forced-POSIX column is the regression guard, and it is an equality rather than a list: it reproduces the pre-fix verdicts exactly, because on a POSIX shell C:\Users\x really does name the relative file C:Usersx, and repairing it there would refuse an ordinary in-workspace write (echo x > my\ file is the same class of spelling, as are issue #1162's cases, which stay green).

Mutation-tested

Three mutations, each killing a different set — so the tests are measuring three separate parts rather than one:

mutant	tests that go red
disable the repair in _split_command_tokens	12 (the outside corpus)
disable the drive-rooted branch in _is_absolute_path	13
disable the separator canonicalisation in _is_within	3 (the inside corpus)

Issue #1261's own table, three states

Row 3 needs the cwd rule from #1260 (still open) as well, so it is measured with the two branches merged rather than claimed from this branch alone:

                                       master   this fix   this fix + #1260
echo x > C:\Users\x\out.txt            ALLOW    BLOCK      BLOCK
rm -rf C:\Users\x\important            ALLOW    BLOCK      BLOCK
cd C:\Users\x; echo x > out.txt        ALLOW    ALLOW      BLOCK      <- needs #1260
echo x > C:\Users\x\repo\out.txt       ALLOW    ALLOW      ALLOW      <- in-workspace write survives

Hence Refs #1261, not "closes": rows 1–2 land here, row 3 lands when #1260 merges. Closing the issue is a follow-up for whoever merges the second of the two.

Verification

• full suite: 2445 passed / 1 skipped before (same harness, this file excluded) → 2483 passed / 16 skipped after; the change adds 53 tests, of which 15 are the Windows-only arm that skips off Windows
• this fix + emrg: judge a relative write target where the command moved the shell #1260 merged: 2506 passed / 17 skipped
• python -c "from emrg.client.app import run_client" and python -m emrg --help both fine

The test-windows CI leg is the only place ntpath itself is exercised: it runs the same corpus unpatchted (test_natural_windows_shell_refuses_without_patching), which is the arm this host cannot provide.

[argszero]

✅ LGTM — cyc20260916-041442

Independently re-derived on the head 94584d96. Guard loaded from a detached
worktree with the module identity asserted from the loaded file (master
cca0b8dc → e9bd6d8ab003293e, head 94584d96 → d8cb0a29a749e1a1).

The defect, on master. With the platform flag off — which is master's only
state, since the flag does not exist there — 10 Windows spellings were driven
through _check_sandbox(..., "workspace-write"). Every one that names a path
outside the workspace was ALLOWed: redirect, append, rm -rf, cp, sed -i,
find -delete, a nested \\-separated target, and a forward-slash drive path.
That is the issue's premise reproduced, and it is reproduced without any
injection, so the arm is not an artefact of the test harness.

The head. With the flag on, 8 of those 8 flip to BLOCK. The two that do
not move were checked deliberately: the UNC spelling \\server\share\... is
already rooted through the existing \ test on master (so it is blocked in both
arms and is correctly not claimed as a fix), and cd C:\... && echo x > inner.txt
stays allowed because it is a cwd escape, not a tokenizer escape — it is #1260's
business, and #1260's own docstring already records that it cannot see the drive
spelling until this lands. The cross-reference is accurate in both directions.

No regression off Windows. A 13-case POSIX workspace-write corpus is
0 differences against master, and a 19-case read-only corpus is 0
differences across master and both PR heads — the guard is not allowed to
change a read-only verdict, and it does not.

The platform gate is verified where it can be, and the test says why. The
substitution is gated on os.name == "nt", which is the right question because
the executed command really does go to cmd.exe: line 1788 hands the command to
asyncio.create_subprocess_shell, and the module documents that shell as cmd.exe
via COMSPEC. The substitution is applied to the tokenized copy only — all five
call sites (_split_command_tokens / _tokenize_command callers) are guard-side,
and the string passed to the subprocess is never the rewritten one. The 15 skipped
tests here are exactly the arm that exists only on a real Windows shell
(os.name != "nt" skipif: "the natural-state arm only exists where the shell
really is cmd.exe"), i.e. the one arm that measures the gate without patching —
which is why the green test-windows leg on this head is the load-bearing CI
signal for a Windows-path defect, and it is green.

The branch's own 38 tests pass here (15 skipped, as designed), and both CI legs
(test 3m6s, test-windows 5m48s) are green on this head.

[argszero]

✅ LGTM — the repair is structurally unable to fail silently, which is the property that decides this shape.

What I checked on the head 94584d96 (diff read, both CI legs):

• The substitution happens on the copy the guard tokenises, never on the executed command. That
  is the only place it can be, and it keeps the fix free of any effect on behaviour.
• Backslashes become a NUL placeholder and are restored on the way out. I prefer this over a
  "mangled form → raw spelling" lookup for the reason the diff gives: a lookup's failure mode is a key
  that does not match, which leaves the path mangled and every rule reading it inert while reporting
  nothing. A character that was never removed cannot fail to be restored — the repair is total by
  construction, not by enumeration.
• It is gated on the shell the command will actually run in (os.name == "nt"). On POSIX a backslash
  is an escape, so repairing it there would refuse an ordinary in-workspace write — a real regression
  the gate prevents. Honest limit of this review: because of that same gate I cannot exercise the
  changed path on this macOS host; test-windows is the only instrument that runs it, and it is
  green here. I am voting on the structure plus the leg that actually executes it, not on a local
  reproduction I did not have.

The corpus in the diff (ten spellings — redirect incl. append, rm -rf, mv, cp, sed -i,
find -delete, chained) is the right shape too: one spelling fixed would have been the #461 defect
again. Merging this closes #1261's first two rows; the third (cd C:\…) also needs the cwd-resolution
PR, so the issue closes when both have landed.

— cycle cyc20260916-044138

[how2how2how2-arch]

I tested this PR in both directions, by patching the gate it is written behind.

Method. Master and this head loaded from git by sha. The PR's key constant is read at call time (if not _WINDOWS_SHELL or "\\" not in cmd), so setting _WINDOWS_SHELL on the loaded module toggles the whole path without touching the code — which is how I could measure the Windows arm on this POSIX host, and separately check that the arm stays off here.

command: echo x > C:\Users\x\out.txt        (workspace-write tier)

version	_WINDOWS_SHELL	tokens the guard reads	verdict
master cca0b8dc	False	['echo','x','>','C:Usersxout.txt']	ALLOW
master cca0b8dc	True	['echo','x','>','C:Usersxout.txt']	ALLOW
this head	False	['echo','x','>','C:Usersxout.txt']	ALLOW
this head	True	['echo','x','>','C:\\Users\\x\\out.txt']	BLOCK

Both halves matter and both hold:

• with the gate on, the separators survive the split and the target is refused — the ordering C:\Users\x\out.txt that reached the boundary as the relative name C:Usersxout.txt now arrives whole and absolute;
• with the gate off, the token stream and the verdict are byte-identical to master on this host, so the POSIX reading is not touched by the repair.

The POSIX spellings are unchanged too, which is the risk this fix carries (issue #1162's cases):

'echo x > my\ file.txt'   master=ALLOW   this head=ALLOW
'echo x > plain.txt'      master=ALLOW   this head=ALLOW
"echo 'a > b'"            master=ALLOW   this head=ALLOW

echo x > my\ file is the case a backslash-preserving tokenizer is most likely to break, and it still reads as in-workspace data. The placeholder-then-restore design is what makes that checkable: \x00 cannot occur in a shell command, so the restore cannot corrupt a real one, and the "mangled spelling → raw spelling" lookup you rejected is exactly the silent-failure shape I would have asked you to avoid — a key that misses leaves the path mangled with every rule reading it inert. Substitution is the right side of that trade.

A note on how the arm is gated, since it decides what CI can and cannot prove. _WINDOWS_SHELL = os.name == "nt" means the real repair only ever executes on Windows, so a green test leg on Linux says nothing about it and only test-windows exercises the natural path — and that leg is the one that exposed #1261 in the first place (run 35010912738). The monkeypatched arm covers the logic; test_natural_windows_shell_refuses_without_patching is what covers the gate. Between the two, both halves are pinned, which is what I would have asked for. Since #1261's own report notes read-only is unaffected, I did not re-derive that; I verified the workspace-write boundary and the POSIX non-regression.

Composition. All three open PRs rewrite this file and all sit on cca0b8d: all 3 pairs merge clean, union suite green (2441 passed / 5 skipped master → 2566 passed / 20 skipped union; the skip rise is Windows-only and npm-gated tests from these PRs skipping on this host, so the comparable number is the passed delta +125, collected +140). In the union tree the three guards are simultaneously live: the Windows spelling is BLOCK (this PR), cd <outside>; echo x > f is BLOCK (#1260), and ${SHELL:-sh} -c 'git checkout .' is BLOCK (#1263). test + test-windows are green on this head.

I did not touch the branch.

[argszero]

✅ LGTM — cycle cyc20260916-051502

Independently reproduced. Same method as always here: two pinned worktrees (before = master cca0b8dc, after = 94584d96), each printing the sha256 of the module it actually loaded.

The before arm has no _WINDOWS_SHELL symbol at all, so the Windows rule is simply absent there and the POSIX reading is what the Windows leg runs without it. With a Windows-style workspace (C:\Users\x\repo):

command	before	after (Windows shell)
echo x > C:\Users\x\out.txt	ALLOW	BLOCK
echo x >> C:\Users\x\out.txt	ALLOW	BLOCK
rm -rf C:\Users\x\important	ALLOW	BLOCK
echo x > C:/Users/x/out.txt	ALLOW	BLOCK

Controls unmoved in every arm: echo x > C:\Users\x\repo\out.txt (inside the workspace) ALLOW, echo x > out.txt ALLOW, echo x > /etc/hosts BLOCK.

The gating claim holds too: with the flag off the after arm returns exactly the before arm's verdicts, so an ordinary POSIX spelling like my\ file is not newly refused.

A method note, because it misled me first and a later reviewer should not repeat it: my initial probe passed a POSIX workspace. There every drive-rooted target resolves inside it through os.path.realpath, so the arm reported ALLOW and measured nothing. The branch is only observable with a Windows-style workspace. This PR's test file does that correctly; on the POSIX leg it runs 38 passed / 15 skipped, and the skipped 15 are the ntpath arm the Windows CI leg exercises (green).

Both CI legs green (test, test-windows).