[Bug]: systemd user unit from t3 service install sets no PATH, so providers in ~/.local/bin are invisible to the service #4913

Activity

1. martinfrancois commented on Aug 5, 2026

   

   martinfrancois

   on Aug 5, 2026

   More actions

   Before someone adds Environment=PATH= to the unit, two things worth checking.

   /proc/<pid>/environ is the environment the process started with, and does not change when the
   process edits process.env. The server edits it: fixPath() in apps/server/src/os-jank.ts
   runs unconditionally at startup, spawns $SHELL -ilc, and merges the login shell's PATH in.
   That code is in 0.0.31, not just main.

   On my host it recovers the directory, starting from exactly the service's exec-time environment:

   $ env -i HOME=/home/u USER=u SHELL=/bin/bash \
       PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin \
       /bin/bash -ilc 'echo $PATH'
   /home/u/.local/share/mise/shims:/home/u/.local/bin:/home/u/bin:/usr/local/bin:/usr/bin:/bin
   

   So ~/.local/bin is on the PATH the server resolves against, even though the unit never set
   it. If the providers were confirmed missing from the UI or from a spawn error in
   boot-service.log rather than from the PATH comparison, then the cause is probably elsewhere
   and this issue's title points at the wrong thing.

   Second, the obvious fix is a trap. A unit-level Environment=PATH= replaces the systemd user
   manager environment for that variable rather than adding to it, so anything arriving via
   ~/.config/environment.d/*.conf or systemctl --user import-environment is dropped. On Ubuntu
   that is how /snap/bin reaches user services, so a snap-installed tool that works today would
   go missing after the next t3 service update.

   Workaround for anyone who is genuinely affected, so nobody has to wait on this. A drop-in
   survives t3 service update rewriting the unit, unlike editing the unit itself:

   mkdir -p ~/.config/systemd/user/t3code.service.d
   cat > ~/.config/systemd/user/t3code.service.d/10-path.conf <<'EOF'
   [Service]
   Environment="PATH=%h/.local/bin:%h/bin:/usr/local/bin:/usr/bin:/bin"
   EOF
   
   systemctl --user daemon-reload
   systemctl --user restart t3code.service

   Check the drop-in applied. /proc/environ is the right tool for this one, since a drop-in sets
   the exec-time environment, which is exactly what that file shows. The PATH the server ends up
   resolving against may be longer, because fixPath() still merges the login shell's entries on
   top of it at startup:

   tr '\0' '\n' < /proc/$(systemctl --user show t3code.service -p MainPID --value)/environ \
     | grep ^PATH=

   Spell out the full value you want, including the system directories, for the
   replace-not-extend reason above. Keep the quotes: systemd expands %h before splitting the
   line on whitespace, so an unquoted value truncates for anyone whose home directory contains a
   space.

   AI Disclosure: Prepared with the assistance of Claude Code (Claude Opus 5). The commands and
   output above were run and observed on my own machine. Reviewed by me before posting.
2. BJSchmidt commented on Sep 23, 2026

   

   BJSchmidt

   on Sep 23, 2026

   More actions

   Confirming a variant of this on t3@0.0.42, where fixPath() does not help: the T3 Connect relay client (cloudflared) in ~/.local/bin is invisible to the systemd service, so the managed tunnel never starts. t3 connect gives no hint.

   What happened

   • Switched from npx t3 serve to the self-contained install, then ran t3 service install.
   • t3 connect only said "T3 Code is already set up to run in the background on this machine." t3 connect status reported Environment link: provisioned and Relay client: available via PATH (~/.local/bin/cloudflared, 2026.5.2, the pinned version).
   • Remote clients could not connect. No cloudflared process was running.
   • The boot-service log only shows the failure after the 10-minute startup retry window:

   [22:13:36.993] WARN (#370): Failed to reconcile T3 Connect desired link on startup
     { message: 'Managed endpoint runtime could not be started.' }
   

   An earlier boot, with the same binary present, logged the underlying reason:

   WARN Failed to reconcile T3 Connect desired link on startup
     providerKind: 'cloudflare_tunnel',
     reason: 'The relay client is not installed.',
   

   The service was restarted several times, each less than 10 minutes apart, so for a while nothing was logged at all.

   Why fixPath() doesn't cover this

   The login shell does include ~/.local/bin (bash -ilc 'echo $PATH' confirms it), and fixPath() runs; its bash: cannot set terminal process group noise is in the log. However, the relay client doesn't read process.env.PATH directly. It resolves cloudflared through Config.string("PATH") (packages/shared/src/relayClient.ts:117-118). In the bundled effect-smol, ConfigProvider.fromEnv() copies process.env when the provider is constructed ({ ...process.env }). If the default provider was first used before fixPath() ran, it still holds the unhydrated PATH. Our reading of the code suggests that is what happens; we didn't verify it at runtime. Either way, the observed result is that providers may resolve through the hydrated process.env, while cloudflared resolution sees the systemd PATH.

   t3 connect checks resolution from the interactive shell, sees cloudflared on PATH, and never installs the managed copy under <T3CODE_HOME>/tools/cloudflared/…. So there is no fallback.

   Workaround (verified)

   Point the service at the binary explicitly with the existing override variable, via a drop-in:

   mkdir -p ~/.config/systemd/user/t3code.service.d
   printf '[Service]\nEnvironment=T3CODE_CLOUDFLARED_PATH=%s\n' "$HOME/.local/bin/cloudflared" \
     > ~/.config/systemd/user/t3code.service.d/cloudflared.conf
   systemctl --user daemon-reload && systemctl --user restart t3code.service

   Within a second of the restart, the service logged Relay client process started, T3 Connect desired link reconciled on startup and Relay client tunnel connection registered, and remote connection from another machine works. This sidesteps the Environment=PATH= replace-not-extend concern raised above, because it doesn't touch PATH.

   Suggestions

   • Resolve cloudflared against the hydrated PATH, for example by reading process.env.PATH after fixPath(), or by having fixPath() run before any ConfigProvider is built.
   • Or: when t3 connect / t3 service install finds cloudflared only via the user's PATH, persist it as T3CODE_CLOUDFLARED_PATH in the unit, or install the managed copy.
   • Surface tunnel failure in t3 connect status and t3 connect (today they report "provisioned" or "already set up" while the tunnel is down), and log the first reconcile failure right away rather than after 10 minutes.

   Environment

   • t3 0.0.42 (self-contained install, ~/.t3/runtime/versions/0.0.42/t3), systemd user service via t3 service install
   • Ubuntu 25.10 x64 (Proxmox kernel 6.17.2-2-pve), Node v26.8.2
   • cloudflared 2026.5.2 at ~/.local/bin/cloudflared
   • Service exec PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin

   ---

   Diagnosed and drafted via t3 triage with Claude Code (Claude Opus 5.5); reviewed by the reporter before posting.
3. valshin commented on Sep 24, 2026

   

   valshin

   on Sep 24, 2026

   More actions

   Confirming on t3@0.0.42, Ubuntu 24.04 x64, headless, systemd user service (Linger=yes). Same symptoms as the comment above: cloudflared in ~/.local/bin, service exec PATH lacks it, environment.cloud.applyRelayConfig fails with Managed endpoint runtime could not be started., and the T3CODE_CLOUDFLARED_PATH drop-in fixed it immediately (tunnel registered within ~1s of restart).

   One extra detail: t3 connect status reports Version: 2026.5.2 for the PATH-resolved binary, but that binary is actually cloudflared 2026.8.2. The relay client's resolve returns the pinned CLOUDFLARED_VERSION for every source (managed, path, override) rather than the version of the executable it found, so the status output is misleading about the version too, not just about the tunnel being up.