Skip to content

Startup walks the whole project tree (node_modules included) twice — ~10 CPU-seconds per launch, unusable under concurrency #1183

Description

@anandgupta42

What happens

Every altimate-code entrypoint (tui, run, serve, acp) walks the entire project tree on startup — node_modules and .git included — looking for mcp.json. On any repo with dependencies installed this costs ~10-13 CPU-seconds per launch, nearly all of it kernel time, fanned out across the runtime's I/O thread pool (one thread per core).

Measured with the shipped release binary (v0.9.7, 16-core M4, this monorepo after bun install):

altimate-code models run in wall user sys involuntary ctx switches
a repo with node_modules 2.15 s 1.75 s 9.36 s 22,439
an empty directory 1.33 s 0.87 s 0.06 s 1,671
same repo, OPENCODE_DISABLE_PROJECT_CONFIG=1 1.35 s 0.92 s 0.33 s 2,107

One ordinary command, one session, ~10 CPU-seconds and ~20k context switches burned before any work happens.

serve in the same repo reaches "listening" having consumed 14.2 CPU-seconds (peak RSS 1.6 GB). In an empty directory the same startup costs 2.4 CPU-seconds.

Why it matters

  • Single sessions: every launch pays it. It is a large part of why the tool feels heavy on a real repo.
  • Concurrency: the walk runs on a thread pool sized to the machine's core count, so each session tries to use many cores at once purely for syscalls. Several sessions at once drive load average and context-switch rate through the roof and make no forward progress. On a 32-vCPU machine, 8 concurrent sessions produced load average 192 and ~385k context switches/s; confining the same 8 processes to 4 cores with taskset dropped that to load 2.1 and ~48k ctx/s and let all 8 progress. More cores made it worse — the signature of a fan-out that scales with core count.
  • On a laptop this shows up as load average 40+ with three sessions open.

Root cause

Glob.Options in packages/core/src/util/glob.ts has no ignore field, so Glob.scan cannot prune anything. Two call sites work around that by filtering the results:

  • packages/opencode/src/altimate/datamate-transport.tsfindAllMcpJsonFiles, run on serve startup
  • packages/opencode/src/mcp/discover.tsdiscoverExternalMcp, run on every config load

Both carry a comment noting the ignore option was dropped. Filtering results does not help: the directories have already been opened and read. Two further unbounded scans exist — **/favicon.* in packages/opencode/src/project/project.ts and **/*.{sql,ddl} in packages/opencode/src/cli/cmd/check.ts.

Component measurement of the two startup scans against this monorepo: 12.93 CPU-seconds (syncDatamateUrlFromVscodeMcp 10.32 s, discoverExternalMcp 2.61 s).

What it is not

Profiling a live process rules out a spin loop. At idle the process holds ~40 threads (16 runtime I/O pool, 16 tokio workers inside the native module, 4 JIT, 3 GC helpers) and consumes 0.008 cores; every pool and tokio thread samples in __ulock_wait2 / _pthread_cond_wait. The threads are parked, not spinning. The CPU is real syscall work — just enormously more of it than intended.

Fix

Restore ignore on Glob.Options and pass it through to glob, which prunes a subtree when the pattern ends in /**, then use it at the four scan sites.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions