Should devlaunch pack the bare caches' refs #465

Activity

1. blooop

   added

   wayfinder:grilling

   on Aug 25, 2026
2. blooop

   added a parent issue on Aug 25, 2026

   • What dl reclaims and what it must leave #444
3. blooop

   marked this as blocking on Aug 25, 2026

   • Write the reclaim contract into docs/cleanup.md #455
4. blooop

   self-assigned this

   on Aug 25, 2026
5. blooop commented on Aug 25, 2026

   

   blooop

   on Aug 25, 2026

   OwnerAuthor

   More actions

   agent-decided (long-term maintainability): yes, on the detached background sweep — one git pack-refs --all in the arm immediately after a successful fetch_all, which is the same lock scope, the same path, and the same shape as the git call already there. Principle 1 is silent, since packing is pure. Principle 2 turns out to be weakly engaged rather than strongly, which the scaling check below is what establishes — so this is decided on principle 3's maintainability clause, over its simplicity clause, and the reason is placement rather than bytes.

   The check first, because it changes the frame

   The ticket asked whether 20.6 MiB is the typical case or the outlier before deciding. It is the outlier, and the mechanism is worth stating precisely: the cost is one filesystem block per loose ref, so it tracks ref count and has essentially nothing to do with repository size. Measured now, cheaply, against real remotes with git ls-remote (no fetches, nothing from #453 re-measured):

   remote	heads	tags	refs	loose refs at 4096 B
   microsoft/vscode	4903	439	5342	20.9 MiB
   torvalds/linux	1	1886	1887	7.4 MiB
   numpy/numpy	34	469	503	2.0 MiB
   blooop/bencher	199	196	395	1.5 MiB
   prefix-dev/pixi	93	280	373	1.5 MiB
   rust-lang/rust	10	324	334	1.3 MiB
   loft-sh/devpod	8	236	244	1.0 MiB
   ros2/ros2	50	120	170	0.7 MiB
   blooop/wayfinder	64	49	113	0.4 MiB
   blooop/devlaunch	30	57	87	0.3 MiB

   torvalds/linux is the point. By history it is larger than everything else here and it carries 1887 refs, barely a third of vscode's — while rust-lang/rust sits at 334. What drives the figure is how many branches upstream leaves open, and vscode's 4903 heads is a hygiene outlier, not a size one. Median of the sample is ~370 refs, about 1.4 MiB; a 20-to-40-repo cache therefore carries something like 30 to 60 MB in total, a couple of percent of one bare's own size.

   Two consequences. First, principle 2 barely bites: loose refs are bounded by the remote's ref count and do not accumulate (heads are pruned, per #453; the tags leak is #453's item 1 and a different ticket's problem), so this is a constant-factor representation cost, not straggler data piling up. Option 3's simplicity claim is genuinely live and I gave it a real hearing. Second, "reclaim 20 MiB" is the wrong headline for the change, so it cannot be what carries it.

   What carries it instead: the work has a home where it costs one call

   fetch_all has exactly one production caller — lazy_fetch, whose only production caller is sweep_repo_fetches. The launch path uses fetch_ref for a single branch and never runs the broad refspec. So "after a successful fetch_all" is the detached sweep today, by construction rather than by convention, and it is the code that creates the loose refs sitting next to the code that packs them. The sweep is already a bounded git invocation on the bare, under run_if_lock_free, with every failure an arm and the loop carrying on. A second bounded git invocation on the bare is not a new kind of thing there; it is the kind of thing that path is.

   And the reclaim is not the only return. commits_beyond_every_ref is rev-list --count <rev> --not --all asked of the bare, and it is #446's reachability proof, run per worktree under a lock (#459 measured 593 ms over 37 clones). On vscode's bare, every one of those probes reads 5286 loose ref files instead of one 426 KB packed-refs. Packing makes this map's own safety machinery cheaper on precisely the repositories where it is slowest, which is a maintainability return that survives the 20-MiB headline collapsing.

   The honest cost, stated rather than waved past. The ticket asks what it means to add work to a path nobody waits on, and the sweep's own doc comment already answers it: the lock this takes is the one a launch blocks on, so "nobody is waiting" is false in exactly the way that matters. The sweep never queues for a launch, but a launch queues for the sweep. So the real figure is 0.36 s worst case added to a scope that is bounded at BACKGROUND_FETCH_TIMEOUT = 300 s and that just spent 0.86 s steady-state (46 s cold) on the fetch. It is a rounding error against the wait it lengthens, and it only ever happens on a pass that actually fetched — a Fetched::Skipped pass adds nothing, and a repeat pack on an already-packed bare is trivial. No threshold and no --auto: pack-refs --all is idempotent, and a loose-ref threshold would be a second thing to maintain and mistune for no measured gain.

   Why not --prune

   Not for the stylistic reason the ticket offers. --prune's architecture is classify directories, print the plan, re-classify each one under the lock, remove: PrunePlan is Reclaimable and Kept, and its whole design point is that a plan and the act on it cannot disagree. Packing refs is not that operation. There is no directory to classify, nothing to weigh with exclusive_usage, nothing for a y/N to be about (packing needs no consent, because it removes nothing), and the bare it would touch is a thing --prune has already decided to keep. PrunePlan has no arm for "mutate something we keep", and adding one would put a verb of a different shape inside the exact structure this map exists to stop from acquiring exceptions. That the bare would be the first bare --prune touches is the symptom; the model mismatch is the reason.

   Why not nowhere

   Because "nowhere" is the right answer when the reclaim has no natural home, or when the safety argument cannot be made, or when the population is somebody else's (the 86.5 GB in #450, the 39 volumes in #451, buildkit's cache). None of that holds here. The remedy is pure, one call, in a path that already does this exact kind of thing under a lock it already holds, and it pays a second time in probe latency. Refusing that is not simplicity, it is leaving a known cost in place to avoid twelve lines.

   Build work implied (not ticketed here, reported to the map)

   1. Git::pack_refs(&self, bare: &Path) in clients/git.rs: git pack-refs --all, cwd the bare, with_timeout(ABOUT_ONE_REPO). Deliberately no --auto (no-op on the files backend, measured both directions in Confirm the bare object stores are bounded #453) and no --prune flag of git's own.
   2. One call site in RepositoryManager::fetch_repo (flows/repo_manager.rs), after the fetch's refusal check and before the last_fetched stamp. A refusal from packing must not become a FetchRepoError: the fetch happened and the stamp has to land, so packing's outcome is its own arm reported as a CacheNotice and never as a failed fetch. An explicit arm, not let _ = — the module's own comment on RecordUpdate::Absent is the precedent.
   3. Tests, as the ticket specifies: the ref set is identical before and after (nothing lost, which is the whole safety claim and is cheap to assert exactly), the packed-refs ref count rises, and the argv reaches the spawn as pack-refs --all with the bare as cwd. Plus one asserting a packing refusal leaves fetch_repo returning Ok with the stamp written.
   4. docs/cleanup.md gains the contract row: loose refs in the bare, packed by the background sweep after each fetch, ~4096 B per ref against ~81 B packed, and the note that pack-refs --auto and gc.auto cannot be leaned on. No em or en dashes (test_docs_prose.py).

   Independent of #453's item 1 (the --tags refspec), and either can land first.
6. blooop

   closed this as completedon Aug 25, 2026