Skip to content

feat: native HTTPS-proxy support and -transport=auto (Meta Muse / proxy-only sandboxes) - #470

Merged
TeoSlayer merged 25 commits into
mainfrom
feat/native-https-proxy
Sep 24, 2026
Merged

TeoSlayer merged 25 commits into
mainfrom
feat/native-https-proxy

Conversation

@TeoSlayer

@TeoSlayer TeoSlayer commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Pilot does not work inside Meta Muse or other hosted agent sandboxes. In a Muse VM:

  • the agent runs as root with HOME=/root and no systemd;
  • outbound traffic is allowed only through an authenticating HTTPS_PROXY that permits CONNECT host:443;
  • UDP is blocked;
  • DNS for *.pilotprotocol.network is poisoned to 198.18.x.x;
  • any connection that bypasses the proxy is killed;
  • the proxy rotates the credentials in HTTPS_PROXY every few minutes. A fresh shell sees the current ones; a long-running process keeps its launch-time copy and gets 407 on new CONNECTs while its old tunnels stay up.

Baseline with released v1.13.9 in a local Muse simulator: install.sh gets through the proxy, but the daemon never registers. It dials 34.71.57.205:9000 directly, which is killed. The only recipe that works today is a root-only SNI router, which needs CAP_SYS_ADMIN.

What changed

Built on github.com/pilot-protocol/common v0.5.15 (netproxy with credential refresh: WithRefreshCommand / WithRefreshFunc / PILOT_PROXY_CMD, rejection (407 or garbled answer) → refresh → retry-once in the Dialer, RefreshingTransport for HTTP; idempotent registry client Close). Includes feat/pilotctl-proxy-passthrough. Merged with current main (#464, #467, #468, #469, #471).

pilot-daemon

  • -proxy <auto|off|URL>, also $PILOT_PROXY and config.json proxy.
    • auto with compat sends the registry TLS, the WSS beacon and every plugin or daemon HTTP client through $HTTPS_PROXY / $ALL_PROXY, honoring $NO_PROXY.
    • Connections CONNECT by host name, so poisoned local DNS does not matter. TLS is end to end.
    • An explicit URL proxies everything except loopback.
    • off, none, no, false and direct all mean off. Any other bare word is rejected instead of being used as a host name.
  • Rotating credentials — -proxy-cmd (also $PILOT_PROXY_CMD, config.json proxy_cmd) drives common v0.5.15's netproxy refresh (netproxy.WithRefreshFunc over proxyconf.CommandSource, which runs the command exactly like WithRefreshCommand — sh -c, own process group killed on timeout, output capped and never logged — but in the environment the daemon was launched with, see the relay below). There is no branch-local refresh logic (internal/proxyconf/policy.go is gone). The command prints the current proxy URL; it runs at startup, again once 60s have passed, and whenever the proxy rejects the credentials — a 407, or a CONNECT answer so garbled it cannot be parsed, which is how Meta Muse's proxy answers them (malformed HTTP status code) — after which that connection is retried once:
    • registry (primary, pool, every redial) and the compat WSS beacon (and every reconnect): netproxy.Dialer;
    • daemon-owned HTTP clients: netproxy.RefreshingTransport (loopback always direct);
    • plugin clients on http.DefaultTransport (configured in place, since plugins use or clone it): https requests are tunnelled through the daemon's proxy relay (below), so a rejected CONNECT, including Muse's garbled form, is refreshed and retried instead of failing, and the error never quotes the proxy's bytes (net/http's own error did: malformed HTTP status code "4O7"). Plain http:// requests keep going to the proxy directly.
    • A failing command keeps the last good URL (at first the launch environment's). Its output is never logged.
  • New: apps follow the rotation too (proxy relay). App-store apps are processes the daemon starts (app-store supervisor, exec with the daemon's environment). Before, they kept the launch-time HTTPS_PROXY: within minutes every app that opened a new connection failed with malformed HTTP status code, also after a respawn — "node online, all apps broken". A daemon that proxies now runs proxyconf.Relay, a CONNECT proxy on 127.0.0.1:<random>:
    • each tunnel is opened upstream by a netproxy.Dialer with the resolver's current credentials (refresh + one retry on a 407 or garbled answer); it never sees inside the tunnel (TLS end to end);
    • CONNECT only, and only with the relay's own credentials (pilot-relay:<random 192-bit token>, constant-time check), so another local user cannot borrow the daemon's proxy credentials; loopback targets go direct, NO_PROXY is honored;
    • a tunnel it cannot open gets 502 (403 if the proxy refused the target, 504 on a timeout) whose reason says why in netproxy's words — never credentials or proxy-supplied text — and a rate-limited WARN with the credential hint;
    • with -proxy-cmd, the daemon points HTTPS_PROXY / https_proxy (and $PILOT_PROXY when it holds a URL) at the relay before any plugin starts, so every app it spawns (and respawns) inherits the relay instead of credentials. HTTP_PROXY is left alone (the relay only tunnels). Without -proxy-cmd the apps' environment is left as it is (it is exactly what the daemon uses);
    • the daemon itself never uses the relay: its refresh command runs in the launch environment, a refresh that would name the relay is refused, and a remote restart (syscall.Exec) re-execs with the launch environment.
  • -transport=auto picks UDP if the beacon answers a UDP discover. Otherwise it picks compat when the compat beacon answers a TLS GET with 426 Upgrade Required. New: it also picks compat when a proxy is configured and it refuses the check (407, 403, a garbled answer, or the proxy is unreachable). Before, auto settled on udp and the daemon dialed the raw registry directly, past the proxy, which Muse kills (E2E scenario 2c). Now every connection stays on the proxy, and the choice is logged at WARN with the proxy's answer and a hint. The registry dial retries with refreshed credentials and, if the proxy keeps refusing, fails with a hint that names the fix. With no proxy involved and nothing reachable, auto still stays on udp. The daemon's own default is still udp. pilotctl daemon start asks for auto only when nothing is configured, and auto is never saved to config.json.
  • New: diagnostics for Muse's rejection form. daemon.ProxyRefusalHint / proxyconf.CredentialHint recognise netproxy's report of an unparseable CONNECT answer (read CONNECT response: malformed HTTP status code (response text withheld)) and say it is how Muse rejects wrong or expired credentials, naming -proxy-cmd. The transport auto-selected WARN and the final registry dial (after 10 attempts) error carry that hint. No hint suggests -transport=udp unconditionally any more (it dials past the proxy, which proxy-only sandboxes kill); an unreachable proxy's hint mentions -proxy=off / -transport=udp only for a host that can reach the internet without the proxy.
  • Fatal startup errors are logged at ERROR. Before, they came out at INFO through log.Fatalf. fix(daemon): graceful watchdog exit (no orphaned apps) + built-in log size cap #464's fatalAfterPluginStart also logs at ERROR now.
  • Precedence for -transport, -proxy, -proxy-cmd, -registry-trust and -registry-fingerprint: flag, then $PILOT_*, then config.json, then the default.
  • Registry routing:
    • When the dial goes through a proxy, the compiled-in raw registry moves to registry.pilotprotocol.network:443 over TLS, in any transport.
    • An explicit -registry-tls=false and a custom registry are kept as they are.
    • Pinned trust is picked up from config or env.
  • The https-proxy TLS uses system roots. The beacon's pinned roots apply to the beacon only.
  • pkg/daemon API: Config.Proxy *netproxy.Resolver is the only knob (the resolver carries the refresh). ResolveProxy(spec, transport, ...netproxy.Option). SelectTransport also returns the proxy error that kept it on compat.

pilotctl

  • daemon start --transport <udp|compat|auto> --proxy <auto|off|URL> --proxy-cmd <command> forwards the proxy and TLS environment. A credentialed proxy URL travels as $PILOT_PROXY and --proxy-cmd as $PILOT_PROXY_CMD; neither goes on argv.
  • New: daemon start says why a start failed. It notices a daemon that exits during startup right away instead of polling until the deadline. On an exit or a timeout it prints the daemon's last error and its last proxy error from the log, with a hint. Example: did not become ready within 30s; last proxy error: proxy CONNECT registry.pilotprotocol.network:443: 407 Proxy Authentication Required, with a hint about credentials and proxy_cmd. Muse's garbled form (... read CONNECT response: malformed HTTP status code (response text withheld)) gets the same credentials/proxy_cmd hint (it used to get "the proxy must allow CONNECT to registry... and beacon..."). Before, it said only "did not become ready".
  • It probes pilot-daemon -help. Flags an older daemon lacks are dropped with a warning, and auto becomes udp. New: a daemon that predates proxy support, started from a shell with HTTPS_PROXY, gets a warning that it will not use the proxy.
  • On a Linux host without systemd where HTTPS_PROXY or https_proxy carries credentials, it sets PILOT_PROXY_CMD=bash -c 'case $https_proxy in *@*) printf %s "$https_proxy";; *) printf %s "${HTTPS_PROXY:-$https_proxy}";; esac' unless a proxy_cmd is already configured: the variable that carries credentials wins ($https_proxy, which Muse's guidance reads from a fresh shell, when both do; the daemon's own order when neither does), so the command can never swap the credentialed URL the daemon started with for one without credentials (the old ${https_proxy:-$HTTPS_PROXY} did, with HTTPS_PROXY=http://u:p@proxy and https_proxy=http://proxy). The ready summary shows where the refresh command came from, but only when the daemon reports that it uses one.
  • lookup, register, rotate-key, the auto-handshake check and recovery recover follow the daemon's network (cmd/pilotctl/registry_dial.go):
    • They use the proxy when the daemon runs compat or an explicit proxy is set.
    • They dial directly on a udp host.
    • When the transport is unknown, they try the proxy first and direct second. The direct attempt is probed, so a sandbox network guard is detected.
    • Whenever a proxy applies, the first route is the TLS registry through it. The raw registry is never dialed directly first (TestRegistryRoutesProxyFirstWhenProxyConfigured).
    • New: they use $PILOT_PROXY_CMD, config proxy_cmd or the sandbox command too, so a stale HTTPS_PROXY in pilotctl's own environment is refreshed and a 407 is retried.
  • config --set transport=|proxy=|proxy_cmd= deletes the key. Switching back to udp restores the raw registry.
  • --json trusted list now returns JSON.

install.sh. This is a byte-identical copy of pilot-protocol/release#49 at c36ad9b (sha256 4384f0cb2890177cb87a2e2bc49e1b814d2850434d24c54d1730c9955fbbb671), and of the website fallback in pilot-protocol/website#263 (a7f94bc). It:

  • adds --transport auto|udp|compat;
  • sends all downloads through the proxy (rotation mid-install: one refresh + retry);
  • allows root on Linux without systemd when root runs it, and refuses root under sudo/doas for a regular user;
  • saves proxy_cmd (the credential-preferring SANDBOX_PROXY_CMD above) in sandboxes whose proxy environment carries credentials, never for credentials that come from PILOT_PROXY and never next to an explicit PILOT_PROXY (release#49 round 2);
  • with a daemon that cannot use the proxy (v1.13.10 and earlier), points proxy-only Linux hosts at the sandbox recipe, never at pilotctl daemon start; restart advice there stops the running daemon first; macOS is never sent to the Linux-only recipe and its LaunchAgent start line carries the same condition;
  • states the transport after the download (and truthfully for --transport auto with a pre-auto daemon);
  • lets --version/--channel beta install again;
  • keeps managed-node mode.

TestInstallerSavesTheSandboxProxyCmd fails if install.sh's SANDBOX_PROXY_CMD ever differs from pilotctl's sandboxProxyCmd (a saved proxy_cmd turns pilotctl's off, so they must not drift).

Docs: README, CHANGELOG, docs/cli-reference.md (regenerated, no change).

Round 3 (review + E2E follow-ups, this push)

  • Installer / pilotctl command drift (review: "the installer users actually run still saves the old proxy_cmd"): release#49 c36ad9b saves exactly sandboxProxyCmd; install.sh here is synced; the new test above pins it.
  • gosec: #nosec G204 with the reason on proxyconf.CommandSource (running the operator's proxy command with sh -c is its purpose), unchecked Close/Remove handled in relay.go and pilotctl; a hint constant renamed so G101 does not read it as a credential. The gosec check passes.
  • Hints fit a daemon already on proxy_cmd (E2E scenario 3): when the daemon re-reads its credentials with a command and the proxy still rejects them, pilotctl daemon start names that command and says to check what it prints from a fresh shell, instead of telling the operator to set one (TestProxyErrorHintWithProxyCommandInUse); the daemon's own CredentialHint is worded for both cases.
  • Flaky test fixed: TestConfigureTransportViaRelayHandlesGarbledRejection ranged over a map and then restored a password named after one iteration; Go's random map order failed it half the time (architecture-gates race run). Fixed order.

Compatibility

  • Defaults unchanged on normal networks. Without a proxy or blocked UDP, a node still runs udp with the raw registry. With HTTPS_PROXY exported on a host where UDP works, the registry and beacon still go direct.
  • Behavior change, only with a proxy configured: auto no longer falls back to udp when the proxy refuses the compat check. A host whose proxy refuses the Pilot hosts and whose UDP is blocked now fails naming the proxy error; it used to start degraded on udp with direct dials. -transport=udp forces the old behavior.
  • New pilotctl with the v1.13.9 daemon works. Unsupported flags are dropped with a warning, and there is a new warning when HTTPS_PROXY is set. With --transport compat, the default --registry is not forced, and v1.13.9 registers in compat.
  • v1.13.9 pilotctl with the new daemon works unchanged.
  • config.json written by the new installer, used with the v1.13.9 daemon: the proxy_cmd key is ignored. auto is never saved, and install.sh --version <old> and pilotctl update --pin rewrite a saved auto to udp.
  • Known low-severity gap: a full downgrade to v1.13.9 pilotctl plus daemon. If config.json has transport=compat and keeps registry=34.71.57.205:9000, the node does not register. update --pin does not rewrite the registry the way install.sh --version does. v1.13.9 cannot work in Muse anyway.

Security

  • Proxy credentials never appear in logs, errors, -help, the daemon argv, the pilotctl output or pilotctl's excerpt of the daemon log. netproxy never quotes a proxy's reply or the credentials; ConfigureTransport turns every refused CONNECT into a netproxy.ConnectError for the same reason, and with the relay a garbled answer no longer reaches net/http's quoting error at all.
  • The proxy relay listens on loopback only, accepts only CONNECT (64 KiB request cap, 30s to send it) and only with its own random credentials (constant-time compare); a client never receives the proxy's credentials, and the relay's token is handed to child processes through their environment only (never logged). Its reason phrases are reduced to printable ASCII and never carry credentials or proxy-supplied text.
  • The output of the refresh command is never logged. netproxy runs the command in its own process group, kills the group on timeout, and caps the output at 64 KiB.
  • Loopback and localhost are never sent to a proxy.
  • TestHelpNeverPrintsProxyCredentials, TestProxyCredentialsWithReservedCharactersStayOffArgv, and password checks in every new test.
  • gosec: the daemon exec, the log read and the proxy command (CommandSource, G204) are annotated with their reasons; unchecked Close/Remove errors in the new code are handled. The gosec check passes.

Tests

All commands run with GOWORK=off, on macOS. Round 3 at head b01d0c45 (main through #468/#469/#471 merged, 0 behind): go test -short -count=1 ./internal/proxyconf ./cmd/pilotctl ./cmd/daemon ./pkg/daemon pass; go test -race -count=5 -run TestConfigureTransportViaRelayHandlesGarbledRejection ./internal/proxyconf pass; gofmt/vet clean. Earlier rounds at 5097bc22:

  • gofmt is clean and go vet ./... is clean; GOOS=linux go vet of the changed packages is clean.
  • go test -count=1 ./cmd/... ./internal/... ./pkg/daemon/... (full, not -short): all 23 packages pass.
  • go test -race -short of internal/proxyconf and cmd/daemon (whole packages) and the proxy/transport/registry/sandbox tests in cmd/pilotctl and pkg/daemon: pass.
  • pilotctl --help capture is unchanged (docs/cli-reference.md needs no regeneration).

New tests for the phase-3 review findings (each fails without its fix):

  • cmd/daemon TestAppsFollowProxyRotationThroughTheRelay: the real daemon (main() in a subprocess) and the real app-store supervisor spawn a sideloaded app (a stdlib net/http client built in the test, like plainweb) behind a proxy that rotates its credentials and garbles stale ones the Muse way. The app's HTTPS_PROXY is the relay; across two rotations every request succeeds and the daemon output holds no password. With the relay disabled the same test fails with malformed HTTP status code "4O7" on every request after the rotation (checked).
  • cmd/daemon: TestAppRelayEnvironment (exported variables, HTTP_PROXY untouched, PILOT_PROXY only when it is a URL, the refresh still runs in the launch environment, a refresh printing the relay is refused), TestRelayExportedOnlyWithProxyCommand, TestProxyErrorHintGarbledRejection, TestAutoTransportGarbledRefusalHintsCredentials (real daemon, wrong password, garbled answers: the WARN hint names the credentials and -proxy-cmd, never -transport=udp; the output never contains 4O7).
  • internal/proxyconf: TestRelayFollowsRotationForApps (3 garbled rotations, one refresh and one retry each), TestRelayRequiresItsCredentials, TestRelayRefusals, TestRelayServesAndLoopGuard, TestRelayCloseEndsTunnels, TestConfigureTransportViaRelayHandlesGarbledRejection (the reviewer's repro: the first GET after a garbled rotation succeeds with exactly 1 refresh, for DefaultTransport and a clone; unrefreshable credentials give the relay's 502 with netproxy's wording; http:// and loopback keep their routes), TestConfigureTransportWithoutRelayGarbled (documents the limit without the relay), TestCredentialHint, TestCommandSourceUsesGivenEnvironment, TestCommandSourceFailures, TestCommandSourceTimeoutKillsTheGroup.
  • pkg/daemon: TestProxyRefusalHintGarbledAnswer.
  • cmd/pilotctl: TestSandboxProxyCmdKeepsTheCredentialedProxy (runs the command through bash for 7 variable combinations, and through a real resolver in the reviewer's case), TestCLIDaemonStartGarbledProxyAnswerHint.

Earlier tests. Each uses in-process rotating-credential CONNECT proxies, which accept one password at a time and answer anything else with 407:

  • internal/proxyconf:
    • TestDialContextRetriesAfterRotation: 407 → the command runs again → one retry; credentials that stay wrong get no retry.
    • TestRoundTripperRetriesAfterRotation: POST with a replayable body, a body that cannot be replayed, loopback direct. The base transport is already configured in place, as http.DefaultTransport is.
    • TestConfigureTransportRefreshesOn407: DefaultTransport and a clone of it.
    • TestResolveWithRefreshCommand.
  • pkg/daemon:
    • TestProxyRefreshCommandFollowsCredentialRotation: real Start in compat through the proxy; after a rotation, the registry reconnect and the WSS beacon reconnect both work.
    • TestProxyRefreshCommandPeriodicRefresh: the timed refresh, 0 × 407.
    • TestNewHTTPClientRetriesAfterRotation: the MOTD client.
    • TestSelectTransportProxyRefusalStaysCompat: 407, 403 and an unreachable proxy all give compat plus the proxy error; the direct path and a proxied beacon outage still give udp.
  • cmd/daemon:
    • TestAutoTransportProxyRefusalStaysCompat: real daemon subprocess with a wrong proxy password. It stays on compat, with a WARN and a hint, and makes no raw-registry CONNECT and no direct dial.
    • TestResolveProxy (-proxy-cmd rules).
    • TestProxyCmd*: subprocess rotation.
  • cmd/pilotctl:
    • TestRegistryDialFollowsRotatedProxyCredentials: stale HTTPS_PROXY → 407 reported; the command supplies current credentials; a rotation between resolve and dial → 407, refresh, retry.
    • TestRegistryRoutesProxyFirstWhenProxyConfigured.
    • TestExtractProxyErrorFromDaemonLog.
    • TestCLIDaemonStartReportsDaemonProxyError: a timeout, and a daemon exit noticed early.
    • TestCLIDaemonStartWarnsOldDaemonIgnoresProxy.
    • TestCLITrustedListJSON.
    • TestE2EDaemonStartBehindRejectingProxy builds the real pilot-daemon. It is skipped under -short, so CI does not run it.

Mutation checks: each of these makes the tests above fail:

  • keeping ProxyFor out of the auto probe (the old udp fallback);
  • replacing RefreshingTransport with a transport configured in place;
  • letting ConfigureTransport's CONNECT hook stay on the RefreshingTransport base.

Binaries: darwin/linux × amd64/arm64 (pilot-daemon, pilotctl, pilot-updater), built with CGO_ENABLED=0 -trimpath -ldflags "-s -w -X main.version=v1.13.9-muse-<sha>".

End-to-end

Round 3, Muse sim (rotating-credential CONNECT-443-only proxy, password rotates every 60s and only the current one is accepted; root, HOME=/root, no systemd, Pilot names poisoned, default route to a packet sink that logs every direct dial; branch binaries at 30c034bb for linux/arm64, installed by release#49 c36ad9b through pilot-skills#34 a366fe8):

Scenario Result
1. One-shot curl … muse/install.sh | bash as root rc=0 in 33s, no manual steps. pilot-up: native path, -transport=compat, proxy credentials: cmd; daemon argv has no credentials; log proxy=auto: http://***@e2e3-proxy:3128 … (credentials refreshed by command); registered 5s after start. config.json: transport=compat, proxy_cmd = sandboxProxyCmd, no registry key.
1. Queries across rotations (gen 1→6, one of each per minute) pilot-mom 5/5, app-store plainweb.fetch 5/5 (the app's proxy is the daemon's relay; every stale-generation 407 was followed by an ALLOW), list-agents 0/5 — the production list-agents problem below, not proxy related.
Skill injection (skillinject#42 tick against pilot-skills#35's manifest) With the marker #34 now writes (skills_dir=/root/workspace/skills, skill_format=muse) the gated muse row is active: drifted → rewrite, and after deletion absent → create, Muse frontmatter.
3. Wrong proxy password during the one-shot muse/install.sh names the 407 and says to rerun from a fresh shell (rc=1). The outer curl -fsSL … | bash still shows only curl: (22) (documented in pilot-sandbox troubleshooting).
4n. Released v1.13.10, root without CAP_SYS_ADMIN pilot-up exit 3 now says v1.13.10 is the latest release and has no -proxy, so upgrading does not help yet (no more loop). The release#49 warning names the recipe.

Phase-3 review fixes (macOS, the reviewer's harness: sandbox-exec profile allowing only localhost egress; a Muse-like CONNECT proxy on 127.0.0.1, :443 only, Basic auth, stale credentials answered with HTTP/1.1 4O7 ...; a "fresh shell" via BASH_ENV; config.json proxy_cmd = the new sandbox command; real registry, beacon, catalogue and plainweb through the proxy):

Scenario Result
Daemon start (auto) compat through the proxy, registered in ~4s; proxy relay listening addr=127.0.0.1:<port>
plainweb installed, plainweb.fetch works; the app's environment (ps -E) is HTTPS_PROXY=http://pilot-relay:***@127.0.0.1:<port>, no proxy credentials
Reviewer's step 6: rotate, kill the app, fetch from the respawned app works — proxy log DENY 407 plainweb-...run.app:443 then ALLOW (the relay refreshed and retried); before the fix this was malformed HTTP status code "4O7"
4 rotations 60s apart, app respawned after each, 3 fetches per rotation 12/12 fetches OK; each first CONNECT after a rotation shows the garbled DENY → ALLOW retry. An earlier 16-round run (8 app kills) was also clean except while the supervisor's crash backoff kept the app down
Wrong password (BASH_ENV=/dev/null), fresh HOME, daemon start last proxy error: ... read CONNECT response: malformed HTTP status code (response text withheld) with hint: "the proxy's answer to CONNECT could not be parsed ... reject wrong or expired credentials ... proxy_cmd"; daemon WARN transport auto-selected hint names the credentials and -proxy-cmd, no -transport=udp; plugin errors via the relay, never 4O7
Logs no proxy password and no relay token in any daemon log
list-agents query fails right now for both the pre-fix (d80ac75e) and the fixed binaries (rekey retransmit gave up ... session desync with node 179172 over the compat relay), while it worked an hour earlier in the reviewer's run: an overlay issue outside these findings, see Status

Phase-2 evidence (Muse sim and macOS substitutes, head 1dd9e8e7):

Scenario Result
1. Muse sim, root, one-shot pilot-skills muse/install.sh Registered through the proxy in 7s with proxy_cmd, and only :443 ALLOW lines in the proxy log. The first list-agents query failed with connection_failed, and the same failure hit v1.13.9 over UDP at the same time. #469 (merged here) fixes the "dropping frame with spoofed source" pong drops seen in that run. Re-run: see Round 3 above (registration and rotation fine; list-agents is a separate production problem).
2. Wrong proxy password, plain pilotctl daemon start (auto) Was the product gap: now fixed. See TestE2EDaemonStartBehindRejectingProxy (real daemon, in-process proxy that answers 407): auto stays on compat, the proxy sees CONNECT registry.pilotprotocol.network:443 and never the raw registry, the daemon makes no direct dial, and pilotctl prints last proxy error: proxy CONNECT registry.pilotprotocol.network:443: 407 Proxy Authentication Required with a hint.
4. Normal network, then branch binaries Passed (udp, direct).
5. UDP blocked, no proxy Passed (auto → compat).
Extra: proxy-only sandbox, plain daemon start Passed (compat through the proxy).
Extra: 4 credential rotations with proxy_cmd Passed (10/10 queries). Now covered in-process by the rotation tests above, on netproxy's refresh.

Status

Ready for review. Required checks (Go ubuntu/macos, Analyze Go, Architecture gates, gitleaks, govulncheck) and gosec pass. install.sh matches pilot-protocol/release fails by design until release#49 merges (it compares against release main); re-run it after that merge.

Open items outside this PR:

  • list-agents reliability (production, not proxy related). In the Muse sim and on a normal-network UDP control, released v1.13.9/v1.13.10 and these branch binaries alike: a fresh identity's first query over compat works, later ones fail with cannot connect to 0:0000.0002.BBE4 (port 1001) or a 30s no-reply, and fresh UDP identities fail even the first. pilot-mom works (5/5 here, 18/19 in phase 3). daemon: path probes and legacy keepalives prove liveness (stop spurious path resets) #469 (in v1.13.10 and in this branch) does not change it. This needs a look at the list-agents node/session path before onboarding can be called clean; it does not block this PR.
  • The daemon's -transport=auto probe sends 2 UDP datagrams to the beacon; in Muse those go around the proxy. The Muse installers save transport=compat (no probe) and pilot-mcp skips its own probe there.
  • In cmd mode the daemon's /proc/<pid>/environ still holds the launch-time HTTPS_PROXY (not argv, not a file).

Deploy order (all six PRs)

  1. installer: transport auto/compat, HTTPS-proxy sandboxes, root without systemd release#49 merge → notify-install-sh-sync dispatches to pilot-protocol/website → sync-install-sh.yml puts install.sh into R2 pilot-release-assets/install.sh (the pilot-release Worker reads R2 per request; no Worker deploy). Check: curl -fsSL https://pilotprotocol.network/install.sh | shasum -a 256 → 4384f0cb2890177cb87a2e2bc49e1b814d2850434d24c54d1730c9955fbbb671 (up to 5 min cache).
  2. install.sh: sync the Pages fallback with pilot-protocol/release#49 website#263 merge (Pages fallback public/install.sh, same bytes) → the sync workflow's drift step goes green.
  3. feat: native HTTPS-proxy support and -transport=auto (Meta Muse / proxy-only sandboxes) #470: re-run install.sh matches pilot-protocol/release, merge.
  4. Tag pilotprotocol v1.13.11 on main (release workflow: binaries, checksums.txt, signed latest.json). v1.13.10 shipped without feat: native HTTPS-proxy support and -transport=auto (Meta Muse / proxy-only sandboxes) #470, so until this tag every install gets a daemon without -proxy. Check pilot-daemon -h lists -proxy and -proxy-cmd.
  5. feat(muse): one-shot Pilot-in-Muse installer and pilot-up.sh TeoSlayer/pilot-skills#34 merge → CI regenerates skills.json/setups.json and skills/pilotctl/SKILL.md, dispatches the website deploy; the Muse one-shot (raw.githubusercontent.com/TeoSlayer/pilot-skills/main/muse/install.sh) is live at merge.
  6. feat(setup): install and start Pilot behind an HTTPS egress proxy (Meta Muse) pilot-mcp#23 merge → release 0.3.0 (behaviour change; last published 0.2.13): bump package.json, package-lock.json, server.json (both fields), .well-known/mcp/server-card.json, src/version.js, src/openclaw-plugin/openclaw.plugin.json, src/openclaw-plugin/evaluate.js and the pinned pilotprotocol-mcp@0.2.13 strings in test/{hermes-setup,openclaw-plugin,native-harness-setup,harness-config-contracts}.test.js (test/release-contract.test.js locks them), move [Unreleased] to [0.3.0], push tag v0.3.0 → publish.yml (tag must equal v+version; lint + tests; npm via OIDC, MCP Registry server.json, ghcr.io/pilot-protocol/pilot-mcp).
  7. Add marker-gated skill targets (gatedTools) and the Meta Muse format skillinject#42 merge → tag v0.2.5 → pilotprotocol follow-up PR go get github.com/pilot-protocol/skillinject@v0.2.5 && go mod tidy (GOWORK=off) → next daemon release (v1.13.12; or fold the bump in before tagging v1.13.11).
  8. inject-manifest: gate Meta Muse under a new gatedTools key TeoSlayer/pilot-skills#35 merge (manifest gatedTools muse row). Released daemons drop the key (CI released-skillinject v0.2.2–v0.2.4). No manifest signing step: DefaultManifestPublicKeyHex is empty; only hosts that installed ~/.pilot/skillinject.pub verify inject-manifest.json.sig.

🤖 Generated with Claude Code

teovl and others added 11 commits September 24, 2026 00:11
…rough

`pilotctl daemon start` could not bring a node online from sandboxes whose
only way out is an authenticating HTTPS proxy (Meta Muse):

- config.json from `pilotctl init` holds the raw-TCP registry default
  34.71.57.205:9000 and daemon start forwarded it as an explicit -registry,
  which stops pilot-daemon from switching to registry.pilotprotocol.network:443
  (TLS) in compat mode. In compat mode the compiled-in registry/beacon
  defaults are now left off argv (a default-equal $PILOT_REGISTRY is dropped
  from the child env too).
- PILOT_TRANSPORT never took effect: the daemon's -transport flag defaults to
  "udp", masking the env var. pilotctl now resolves --transport, then
  $PILOT_TRANSPORT, then config "transport" and passes an explicit -transport.

New: --transport <udp|compat> and --proxy <auto|off|URL> (plus config keys
"transport"/"proxy", validated by `config --set`). Both reach the daemon only
when set; a pilot-daemon whose -help lacks them gets them dropped with a
warning instead of a flag-parse crash. A proxy URL with credentials travels
as $PILOT_PROXY (never argv, PILOT-290) and is redacted in config output.
The child env explicitly keeps HTTPS_PROXY/HTTP_PROXY/ALL_PROXY/NO_PROXY
(both cases), PILOT_PROXY, PILOT_TRANSPORT, SSL_CERT_FILE, SSL_CERT_DIR on
both the fork and --foreground paths. --compat-beacon, --registry-trust,
--registry-fingerprint, --tls-trust and the documented-but-dropped
--endpoint/--motd-* are forwarded when given.

Also: create ~/.pilot before the O_EXCL PID claim (a fresh HOME failed with
"PID file locked" forever), and registry dial failures behind a proxy now say
pilotctl's direct registry dials do not use the proxy yet (TODO(netproxy)).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
install.sh --transport compat (or PILOT_TRANSPORT=compat) merges
"transport": "compat" and "proxy": "auto" into ~/.pilot/config.json via
`pilotctl config --set` (atomic, 0600, other keys kept) and generates compat
systemd/launchd units (-transport compat, raw-TCP registry/beacon defaults
left to the daemon). A re-run without the flag keeps a compat config.

Audited for a no-root, no-systemd VM whose only egress is an authenticating
CONNECT-:443 proxy with poisoned local DNS for *.pilotprotocol.network: every
network call is curl over HTTPS (proxy env honored, CONNECT by hostname, no
wget / raw IP / non-443 / registry or beacon probes). Proxy URLs are only
ever printed redacted. Download failures now name the proxy and the hosts it
must allow instead of falling silently into "Go is required". With an older
release (pilotctl without --transport) compat points config's registry at
registry.pilotprotocol.network:443, and a pilot-daemon without -proxy gets
an explicit warning when HTTPS_PROXY is set. No-systemd hosts get the
`pilotctl daemon start` hint; service-manager units get a note that they do
not inherit the shell's HTTPS_PROXY.

Tested in docker (debian bookworm, non-root, temp HOME) on an --internal
network whose only egress is an authenticating CONNECT-443 proxy, with
poisoned /etc/hosts for the Pilot names: v1.13.9 download + SHA-256 verify +
install succeeds; missing/wrong proxy credentials fail with a clear hint.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Hosted agent sandboxes such as Meta Muse block outbound UDP, poison
local DNS for *.pilotprotocol.network, and allow only an authenticating
HTTP proxy that CONNECTs to :443. Until now the daemon ignored
HTTPS_PROXY entirely, so compat mode needed root, an SNI router and a
mount-namespace /etc/hosts override to get online.

New flag -proxy (env PILOT_PROXY, config key "proxy"), resolved once at
startup by daemon.ResolveProxy into a common/netproxy policy:

  auto (default)  -transport=compat: HTTPS_PROXY/https_proxy, falling
                  back to ALL_PROXY/all_proxy, honoring NO_PROXY.
                  -transport=udp: no policy, dialing exactly as before.
  off             never proxy (HTTP clients also stop following env).
  http(s)://URL   that proxy for every outbound TCP/HTTP connection.

The policy (daemon.Config.Proxy) is applied to every outbound
connection: the registry client (primary, pool and reconnects, via
registry.WithDialer), the compat WSS beacon (wss.Config.Proxy), the
MOTD fetch, and http.DefaultTransport for plugin HTTP clients
(catalogue pins, skillinject, trustedagents, webhook, enterprise
control). Targets are CONNECTed by host name and never resolved
locally; TLS, SNI and pinned-fingerprint checks stay end to end.
One startup line logs the transport and the redacted proxy.

Compat mode now treats the compiled-in raw-TCP registry default
(34.71.57.205:9000, which pilotctl passes on every daemon start) as not
explicit, so it still switches to registry.pilotprotocol.network:443.
-transport honors PILOT_TRANSPORT. -beacon-rtt-probe is skipped in
compat mode (its UDP probes cannot leave the host).

Pins github.com/pilot-protocol/common to the netproxy feature commit
(v0.5.14-0.20260923210943-65ea5b1a3d2d); move to a tagged release
before merge.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Replace the feature-branch pseudo-version with the tagged release that
ships netproxy (HTTP CONNECT dialer, independent proxy env parsing,
credential-safe URL handling) and registry/client WithDialer.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…nboarding

Unifies the daemon (-proxy) and pilotctl/install.sh (--transport/--proxy)
branches and fixes every confirmed review finding on both.

pilot-daemon
- -transport=auto (also $PILOT_TRANSPORT, config "transport"): udp when
  the beacon answers a UDP discover (one round trip; two attempts within
  1.5s), else compat when the compat beacon accepts TCP through the proxy
  compat would use, else udp as before. Logged once. Never picks compat
  for a private registry/beacon. The binary default stays udp.
  pkg/daemon exports SelectTransport/NormalizeTransport; the embedder
  path (TransportMode "" or "auto") uses the same check.
- Precedence for -transport, -proxy, -registry-trust,
  -registry-fingerprint: flag, then $PILOT_TRANSPORT / $PILOT_PROXY /
  $PILOT_REGISTRY_TRUST / $PILOT_REGISTRY_FINGERPRINT, then config.json,
  then default. Literal flag defaults: -help never prints $PILOT_PROXY.
- registry_trust/registry_fingerprint from config/env survive compat
  mode; a fingerprint alone selects pinned trust. compat keeps the raw
  registry with an explicit -registry-tls=false (TCP/9000 fallback) and a
  custom registry from config.json. registry.pilotprotocol.network:443
  always gets TLS, also in udp mode.
- internal/proxyconf (on common/netproxy): off also accepts
  none/no/false/direct; bare words and scheme-less values are errors, not
  proxy host names. Loopback targets are never proxied (DefaultTransport,
  daemon HTTP clients, registry and WSS dials), even with an explicit URL.
- WSS beacon dials through a netproxy dialer (wss.Config.DialContext
  replaces Config.Proxy): an https:// proxy is verified with the system
  roots; the beacon TLS config applies to the beacon only.
- Unusable HTTP_PROXY/ALL_PROXY no longer drop a valid HTTPS_PROXY
  (common v0.5.14); skipped variables are logged. Certificate errors
  against the system store name SSL_CERT_FILE and the fingerprint.

pilotctl
- daemon start asks a daemon that supports it for -transport=auto when
  no transport is configured; the daemon is probed once (-help) and
  flags/values it predates are dropped (auto -> udp) with a warning. The
  ready summary reports the transport the daemon chose.
- --proxy, then $PILOT_PROXY, then config "proxy"; any '@' means
  credentials, which travel as $PILOT_PROXY only, and nothing on argv
  contradicts them. Redaction/validation via internal/proxyconf.
- lookup/register/rotate-key, the auto-handshake visibility check and
  recovery dial the registry through the egress proxy (CONNECT by name),
  using registry.pilotprotocol.network:443 over TLS when proxied or in
  compat mode (pinned with registry_fingerprint when configured), with a
  one-shot TLS fallback when the raw registry is unreachable directly.
- config --set transport accepts auto and normalizes; leaving compat
  restores the raw-TCP registry an older compat install saved.
- withTempHomeFull clears PILOT_TRANSPORT/PILOT_PROXY/*_PROXY.

install.sh
- --transport auto|udp|compat; fresh installs save auto when the daemon
  supports it; no "proxy" key is written; units take the transport from
  config.json; --transport udp restores the raw registry.
- Root is allowed in a Linux container/VM without systemd (hosted agent
  sandboxes); regular hosts still refuse without PILOT_ALLOW_ROOT.

Docs: README compat/proxy section (precedence, SSL_CERT_FILE, pinned
registry), env table, CHANGELOG, regenerated docs/cli-reference.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…review

Fixes the six re-review findings on feat/native-https-proxy.

muse-proxy-cred-rotation-unhandled (high)
- New -proxy-cmd / $PILOT_PROXY_CMD / config.json proxy_cmd: a command
  whose stdout is the current proxy URL. internal/proxyconf.Policy wraps
  it: re-run every 60s (Config.ProxyRefreshInterval) and whenever a
  CONNECT gets 407 (or the malformed status line sandbox proxies send),
  then the dial is retried once with the new URL. The registry client
  (primary, pool, every redial), the WSS beacon (every reconnect),
  daemon HTTP clients (retrying RoundTripper) and http.DefaultTransport
  (refresh on 407 via OnProxyConnectResponse) all follow it. Output and
  stderr are never logged; PILOT_ADMIN_TOKEN/PILOT_WEBHOOK_SECRET are
  kept from the command. A failing first run falls back to the launch
  environment's proxy.
- pkg/daemon: Config.ProxyPolicy (*ProxyPolicy), StaticProxyPolicy,
  NewCommandProxyPolicy; Start runs the refresher until Stop.
- install.sh saves proxy_cmd=bash -c 'printf %s "${https_proxy:-$HTTPS_PROXY}"'
  in a Linux container/VM without systemd whose HTTPS_PROXY carries
  credentials (or PILOT_PROXY_CMD when set); README documents it.

version-skew-config-transport-auto-bricks-older-daemon (medium)
- install.sh never saves transport=auto (--transport auto removes a saved
  transport); service units get Environment PILOT_TRANSPORT_DEFAULT=auto,
  which a pre-auto daemon ignores. pilot-daemon honours
  $PILOT_TRANSPORT_DEFAULT only when flag, $PILOT_TRANSPORT and config.json
  choose nothing.
- install.sh --version <pre-auto tag> and pilotctl update --pin rewrite a
  saved transport=auto to udp. pilotctl config --set transport= (proxy=,
  proxy_cmd=) removes the key.

compat-explicit-registry-tls-now-pinned-fatal (low)
- applyRegistryDefaults defaults trust (pinned with a fingerprint, else
  system) whenever TLS is on in compat or for the compat registry
  address, also when -registry-tls was explicit.

auto-compat-check-tcp-only-fatal-on-beacon-outage (low)
- SelectTransport's compat check now does TLS + GET of the beacon path
  and requires 426 Upgrade Required (live WebSocket endpoint); a TCP front
  with the beacon down (502/503/close) keeps auto on udp.

daemon-proxied-udp-registry-stays-raw-9000 (low)
- When the registry dial goes through the proxy (any transport), the
  compiled-in raw registry moves to registry.pilotprotocol.network:443
  over TLS, the rule pilotctl already applied.

pilotctl-auto-proxy-regardless-of-transport (low)
- pilotctl's registry routes follow the daemon: an explicit proxy URL
  always; the environment's proxy only when the transport is compat (the
  running daemon's, from the new info "transport" field, else
  $PILOT_TRANSPORT / config.json). udp hosts dial directly; unknown
  transport tries direct first and the proxied TLS registry as fallback
  for the production registry only; private raw registries are never sent
  to the environment's proxy outside compat.

Tests: proxyconf policy unit tests (407 retry for dials and HTTP, NO_PROXY,
Run loop, command runner); pkg/daemon compat start through a rotating
proxy (registry + WSS reconnect); cmd/daemon end-to-end runs for
PILOT_PROXY_CMD rotation, command failure fallback, proxied udp registry,
explicit -registry-tls trust, PILOT_TRANSPORT_DEFAULT; SelectTransport
front-up/beacon-down; pilotctl route matrix, private registry direct,
fitTransportToDaemon, config key clearing.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…tl registry dial

Fixes the two re-review findings on feat/native-https-proxy.

web4-install-sh-edits-not-in-canonical-installer (high)
- install.sh here is a synced copy of pilot-protocol/release:install.sh
  (the script https://pilotprotocol.network/install.sh serves; the
  canonical-drift job enforces byte equality). The installer changes are
  now ported onto the canonical script in pilot-protocol/release branch
  feat/installer-proxy-transport, keeping its managed-node mode
  (--managed-url, --no-start, PILOT_ENROLLMENT_TOKEN), and this copy is
  byte-identical to that branch. canonical-drift passes once the release
  change merges; merge it first.
- Rotation no longer depends on the installer: on Linux without systemd,
  when $HTTPS_PROXY / $https_proxy carries credentials, the proxy setting
  is auto, no proxy_cmd is configured ($PILOT_PROXY_CMD, config.json /
  --config file) and the daemon supports -proxy-cmd, `pilotctl daemon
  start` hands the daemon
  PILOT_PROXY_CMD=bash -c 'printf %s "${https_proxy:-$HTTPS_PROXY}"'.
  A node set up by the currently served installer (PILOT_ALLOW_ROOT=1, no
  proxy_cmd) therefore re-reads rotated credentials too. The ready summary
  says so (JSON: "proxy_cmd").
- README/CHANGELOG describe both paths.

web4-pilotctl-raw-route-defeated-by-muse-guard (low)
- With the transport unknown (no daemon answering, nothing configured) and
  a proxy in the environment, pilotctl now tries the TLS registry through
  the proxy first and the raw registry directly second. The direct
  fallback is probed: a peer that sends data or hangs up within 300ms of
  connecting (a sandbox network guard) is rejected, so the proxied route's
  error is reported instead of a broken pipe. NO_PROXY exempting the
  registry keeps direct-first.

Tests: route plan table (proxy first, probe flag, NO_PROXY), probedDirectDial
(guard that talks, guard that closes, silent registry usable, dial error),
dialRegistry against a local guard + authenticating CONNECT proxy + pinned
TLS registry (proxy allowed, proxy refusing, raw fallback to a real
registry), sandboxProxyCmdFor matrix, CLI daemon start passes/keeps
PILOT_PROXY_CMD. Reviewer repro (muse-sim-vm, --network none, guard on
34.71.57.205:9000): CONNECT registry.pilotprotocol.network:443 is now the
first attempt and the error names the proxy.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Comment thread cmd/pilotctl/daemon_transport.go Fixed
Comment thread cmd/pilotctl/daemon_transport.go Fixed
Comment thread cmd/pilotctl/main.go Fixed
Comment thread cmd/pilotctl/main.go Fixed
Comment thread cmd/pilotctl/registry_dial.go Fixed
Comment thread cmd/pilotctl/registry_dial.go Fixed
Comment thread cmd/pilotctl/registry_dial.go Fixed
Comment thread pkg/daemon/transport_auto.go Fixed
teovl and others added 2 commits September 24, 2026 04:35
…roxy

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… on the proxy; daemon start names the proxy error

Credential refresh now comes from common v0.5.15's netproxy instead of the
branch-local proxy policy (internal/proxyconf/policy.go is gone):

- -proxy-cmd / $PILOT_PROXY_CMD / config.json proxy_cmd build the
  resolver with netproxy.WithRefreshCommand. The command runs at startup,
  again once 60s have passed (lookup-driven), and whenever a proxy answers
  407; the rejected connection is then retried once.
- Registry (primary, pool, every redial) and the compat WSS beacon (and its
  reconnects) dial through netproxy.Dialer; daemon-owned HTTP clients use
  netproxy.RefreshingTransport (proxyconf.RoundTripper, loopback direct);
  http.DefaultTransport is configured in place so plugin clients take the
  current credentials and a 407 refreshes them for the next request.
- pkg/daemon: Config.Proxy is the only knob (ProxyPolicy,
  ProxyRefreshInterval, StaticProxyPolicy, NewCommandProxyPolicy removed);
  ResolveProxy takes netproxy options.
- pilotctl daemon start --proxy-cmd (passed as $PILOT_PROXY_CMD, never on
  argv); pilotctl's own registry commands use $PILOT_PROXY_CMD, config
  proxy_cmd or the sandbox default too, so a stale HTTPS_PROXY in its
  environment is refreshed and a 407 is retried.

E2E product gap (phase-2 scenario 2c): with a proxy configured and
-transport=auto, a proxy error during the compat check (407, 403, garbled
answer, proxy unreachable) no longer falls back to udp, which dialed the
raw registry directly past the proxy. SelectTransport picks compat and
returns the proxy error; the daemon logs it at WARN with a hint, and the
registry/compat dial failures end with a hint naming the fix. Fatal
startup errors are logged at ERROR (log.Fatalf printed them at INFO).

pilotctl daemon start notices a daemon that exits during startup instead
of polling until the deadline, and on exit or timeout prints the daemon's
last error and last proxy error from its log with a hint, instead of only
"did not become ready".

gosec: annotate the daemon exec and log read, handle probe Close errors.

Tests: in-process rotating-credential CONNECT proxies for proxyconf
(dial retry, RoundTripper retry, DefaultTransport refresh), pkg/daemon
(registry + WSS reconnect after rotation, timed refresh, MOTD client
retry, auto stays compat on 407/403/unreachable), cmd/daemon (subprocess:
auto + 407 stays on the proxy, mutation-checked), pilotctl (registry dial
follows rotated credentials, proxy-first routes, daemon start failure
reporting with fake daemons, and an end-to-end run of the real daemon
behind a rejecting proxy).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Comment thread cmd/pilotctl/main.go Fixed
teovl and others added 6 commits September 24, 2026 05:04
…re HTTPS_PROXY

Two phase-2 E2E onboarding papercuts:

- `pilotctl --json trusted list` ignored --json and printed the table.
- A new pilotctl paired with a daemon that predates -proxy (v1.13.9) in a
  shell with HTTPS_PROXY/ALL_PROXY started it silently; the failure then
  surfaced only as "did not become ready". daemon start now warns that the
  daemon will not use the proxy (not with --proxy off, and not twice when
  an explicit --proxy was already dropped).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… into feat/native-https-proxy

Conflicts:
- cmd/daemon/main.go: one fileConfig for both flagSources (recorded
  before ApplyToFlags) and #464's flagExplicit; logging.Setup stays early
  (the transport/proxy resolution logs), so #464's logcap.Watch moves up
  with it; the daemon start failure keeps #464's fatalAfterPluginStart.
- cmd/daemon/shutdown.go: fatalAfterPluginStart logs at ERROR (slog), like
  the branch's fatalf, so pilotctl daemon start can report it.
- CHANGELOG.md: both Fixed lists kept.
- go.mod: tidy (golang.org/x/net indirect again, as on main, now that the
  branch-local refresh policy is gone).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
TestStartCompatModeThroughConnectProxy failed on ubuntu CI with
"beacon authenticated node 0, want 1": the daemon's Start returns as soon
as it reads auth_ok, and the fake beacon stored the node only after
writing it. Store first.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ctl update status) into feat/native-https-proxy

Conflicts:
- cmd/pilotctl/updates.go: #468's result fields and printUpdateResult,
  plus the branch's fitTransportToDaemon note for update --pin (JSON
  "note", text "Note:"), with a test that the two coexist.
- CHANGELOG.md: both Fixed lists kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ay; Muse rejection diagnostics; credential-preferring sandbox command

Phase-3 review findings on #470:

web4-470-apps-inherit-stale-proxy-creds (high). App-store apps are
processes the daemon spawns with its environment, so they kept the
launch-time HTTPS_PROXY credentials: after one rotation every app that
opened a new connection failed ("node online, all apps broken"), also
after a respawn. New internal/proxyconf.Relay: a CONNECT proxy on
127.0.0.1:<random> that opens each tunnel upstream with a netproxy.Dialer
(current credentials, refresh + one retry on a 407 or a garbled answer),
CONNECT only, guarded by its own random credentials, never inside the TLS
tunnel. A proxying daemon runs one; with -proxy-cmd it points
HTTPS_PROXY/https_proxy (and PILOT_PROXY when it is a URL) at it before
any plugin starts, so every app it spawns inherits the relay instead of
credentials. The daemon itself never uses the relay: the refresh command
now runs in the launch environment (proxyconf.CommandSource, same
process-group/timeout/output-cap rules as netproxy's), a refresh that
names the relay is refused, and a remote restart re-execs with the launch
environment.

web4-470-configuretransport-ignores-garbled-rejection. net/http never
passes an unparseable CONNECT answer to OnProxyConnectResponse, so
http.DefaultTransport clients never refreshed on Muse's rejection form and
quoted the proxy's bytes. ConfigureTransport now takes the relay: https
requests tunnel through it (refresh + retry, netproxy's wording in errors,
the relay's upstream error surfaced to the client); http:// keeps going to
the proxy. Doc comments and CHANGELOG corrected.

web4-470-muse-rejection-diagnostics-misdirect. proxyconf.CredentialHint /
UnreadableConnectReply recognise netproxy's "read CONNECT response: ...
(response text withheld)"; daemon.ProxyRefusalHint, the auto-selected WARN
and pilotctl's start-failure hint now name the credentials and proxy_cmd
for it. No hint suggests -transport=udp unconditionally any more.

web4-470-sandbox-cmd-variable-precedence. The sandbox proxy_cmd (pilotctl
and install.sh) now prints whichever of $https_proxy / $HTTPS_PROXY
carries credentials ($https_proxy when both do, the daemon's order when
neither does), so it can never replace a credentialed URL with a bare one.
install.sh differs from release#49 by that one line; #49 must take it.

Tests: real daemon + app-store supervisor + sideloaded app across garbled
rotations (fails with "malformed HTTP status code 4O7" without the relay),
relay unit tests, the reviewer's ConfigureTransport repro, CommandSource,
hints (daemon subprocess, pkg/daemon, pilotctl CLI), and the sandbox
command through bash for 7 variable combinations.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Comment thread internal/proxyconf/command.go Fixed
Comment thread internal/proxyconf/relay.go Fixed
Comment thread internal/proxyconf/relay.go Fixed
Comment thread internal/proxyconf/relay.go Fixed
Comment thread internal/proxyconf/relay.go Fixed
Comment thread internal/proxyconf/relay.go Fixed
Comment thread internal/proxyconf/relay.go Fixed
Byte-identical copy of the canonical installer on
pilot-protocol/release feat/installer-proxy-transport, so canonical-drift
passes once release#49 merges. Takes this branch's credential-preferring
sandbox proxy_cmd, and adds: no raw-IP registry/beacon in config.json for
compat or auto behind a proxy, downloads retried once after a proxy
credential rotation, proxy_cmd saved for older daemons too, PILOT_PROXY
downloads, the v1.13.x proxy warning pointing at the pilot-sandbox recipe,
and onboarding text that reads replies from send-message --wait and drops
routine appstore --force. Only install.sh changes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Byte-identical to release@67e83fc (review fixes: root refused under sudo,
no `pilotctl daemon start` advice on proxy-only hosts with a daemon that
cannot use the proxy, the transport stated after the download, --version /
--channel beta installable again).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…all.sh sync with release#49 (c36ad9b)

- pilotctl daemon start: when the daemon re-reads its credentials with a
  proxy command and the proxy still rejects them, the hint names the command
  in use and says to check what it prints from a fresh shell, instead of
  telling the operator to set one (E2E scenario 3: the 407 hint said "give
  the daemon a command that prints the current proxy URL" with config.json
  proxy_cmd in place). The daemon's own CredentialHint is worded for both.
- TestInstallerSavesTheSandboxProxyCmd: install.sh's SANDBOX_PROXY_CMD must
  evaluate to pilotctl's sandboxProxyCmd. A saved proxy_cmd turns off the one
  `daemon start` hands the daemon, so the two must never drift (web4-470
  review: the installer kept the pre-change command).
- gosec: #nosec G204 with the reason on proxyconf.CommandSource (the
  operator's proxy command run with sh -c is the function's purpose), and
  the new G104 warnings (unchecked Close/Remove) in relay.go and pilotctl.
- install.sh: byte-identical to pilot-protocol/release#49 c36ad9b (sandbox
  proxy_cmd only for credentials a fresh shell sees and never over an
  explicit PILOT_PROXY; restart advice stops the running daemon before the
  sandbox recipe; macOS LaunchAgent line conditioned and never sent to the
  Linux-only recipe; truthful --transport auto note for pre-auto daemons).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Comment thread internal/proxyconf/proxyconf.go Fixed
teovl and others added 2 commits September 24, 2026 12:22
… credential

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
TestConfigureTransportViaRelayHandlesGarbledRejection ranged over a map of
the two transports and afterwards restored the proxy to "p-clone", the
password the clone iteration rotates to. Go randomizes map order, so half the
runs restored a password the relay no longer held and the 403 check saw a
garbled 407 instead (architecture-gates race run on 30c034b). Iterate in a
fixed order. -race -count=5 passes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@TeoSlayer
TeoSlayer enabled auto-merge (squash) September 24, 2026 10:26
@TeoSlayer
TeoSlayer merged commit 6d3e807 into main Sep 24, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants