Skip to content

[Bug]: SSH environment starts a second server on the same base dir; a serve crash (ECONNRESET) kills every agent #17846

Closed as not planned

Description

@loutab4k

Before submitting

Area

apps/server (service launcher, SSH environment launch)

Setup

  • Remote Linux VM (Ubuntu, KVM, 4 vCPU, 8 GiB RAM), T3 Code installed as the user systemd service t3code.service (KillMode=mixed, Restart=always), nightly channel, continueThreadsAfterServerUpdate = true.
  • macOS desktop app connects through an SSH environment; T3 Connect is enabled.
  • Versions on the VM: ExecStart pinned to 0.0.46-nightly.20261008.2819 t3 __service-launcher; active serve 0.0.46-nightly.20261009.2873 (port 3773); a second serve 0.0.46-nightly.20261010.2922 (port 3774, see below). Desktop app version: not recorded.

What happened

  1. Two servers on one base dir. While t3code.service was healthy on 127.0.0.1:3773, the SSH environment started its own detached t3 serve --host 127.0.0.1 --port 3774 --base-dir ~/.t3 (ppid 1, in the SSH login session scope, not in the service). Both servers ran on the same ~/.t3 / statev2.sqlite, each with its own cloudflared tunnel. server-runtime.json was overwritten to point at the new one. The service's serve kept running its background loops on the shared DB. The same pattern happened on three days in a row (sshd logged forwards to :3774 on two earlier days).
  2. Duplicate agent sessions. Each server resumed the same Claude session for one thread (claude --resume=<same id> under both servers, same worktree). The session transcript contains one corrupted line (two JSON records fused) around that time; we could not prove which writer caused it.
  3. Crash on ECONNRESET. The service serve crashed with an unhandled stream error, logged in boot-service.log:
    node:events:505
          throw er; // Unhandled 'error' event
    Error: read ECONNRESET
        at Pipe.onStreamRead
      code: 'ECONNRESET'
    [service-launcher] Active child exited unexpectedly (1).
    
    No OOM, no cgroup memory events, memory PSI ~0 (Node v26.8.2).
  4. Any child exit takes every agent down. The launcher's #handleExit throws Active child exited unexpectedly, the service exits 1, and systemd kills the whole cgroup (Killing process … (claude) with signal SIGKILL). Up to 2873 all agent processes ran inside t3code.service, so every serve exit killed all running agents and their background work. A second instance of this: an agent, seeing two servers, sent SIGTERM to the service's serve (exit 130 → same full restart). 2922 moves agents into app-t3code-agents.slice scopes; whether they are re-adopted after a serve crash is unknown.
  5. t3 service status (run with the 2922 CLI) reports "needs an update or repair" for a healthy service whenever the CLI is newer than the active service version.

Expected

  • An SSH environment should use (or update) the running service instead of starting a second server on the same base dir; rejectRunningServer should prevent it.
  • A stream error such as ECONNRESET on a child pipe should not crash the server.
  • A serve exit should not make systemd kill all agent processes.
  • t3 service status should distinguish "older than this CLI" from "broken".

Impact

Over three days, background agent work was cancelled 8 times ("T3 server restarted… background work was cancelled"); at least 2 of those were the crash/launcher path above, 3 were self-updates.

Activity

  1. juliusmarminge commented on Oct 11, 2026

    @juliusmarminge
    Member

    Note

    Grok responding on behalf of Julius.

    Thanks for writing this up in detail! Each part is already tracked separately: the duplicate server / shared base dir in #5749, the serve crash in #16794 / #16555, and the rest in #17662. Closing this in favor of those so the discussion stays in one place. Please add any new repro details there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions