installer: transport auto/compat, HTTPS-proxy sandboxes, root without systemd

[TeoSlayer]

What

This PR fixes the installer users actually run, curl -fsSL https://pilotprotocol.network/install.sh | sh, for hosted agent sandboxes. The target is Meta Muse, where:

• the agent runs as root in a VM without systemd;
• the only way out is an authenticating HTTPS CONNECT proxy on :443;
• DNS for the Pilot hosts is poisoned;
• the proxy credentials in HTTPS_PROXY rotate every few minutes.

Normal hosts install exactly as before (same files, same config.json). Their output now states the transport the installed daemon actually uses.

It is the installer half of pilot-protocol/pilotprotocol#470 (feat/native-https-proxy). At head c36ad9b this branch's install.sh (sha256 4384f0cb2890177cb87a2e2bc49e1b814d2850434d24c54d1730c9955fbbb671) is byte-identical to pilotprotocol#470's copy (synced in pilotprotocol@df090a8a, where TestInstallerSavesTheSandboxProxyCmd also pins SANDBOX_PROXY_CMD to pilotctl's command) and to the website fallback in pilot-protocol/website#263 (website@a7f94bc).

How it deploys (checked)

• Serving. worker/ (Cloudflare Worker pilot-release, route pilotprotocol.network/install.sh) reads install.sh from the R2 bucket pilot-release-assets on every request (cache-control: public, max-age=300). The script is not bundled into the Worker at wrangler deploy, so the Worker does not need redeploying for this change.
• Upload. R2 is written by pilot-protocol/website .github/workflows/sync-install-sh.yml. That workflow:
  1. fetches raw.githubusercontent.com/pilot-protocol/release/main/install.sh;
  2. runs bash -n on it;
  3. runs wrangler r2 object put pilot-release-assets/install.sh;
  4. diffs the live URL against it;
  5. finally fails if the website's own fallback public/install.sh has drifted.
• Trigger. This repo's notify-install-sh-sync.yml runs on a push to main that touches install.sh. It sends repository_dispatch: release-install-sh-changed to the website repo. A daily 06:17 UTC cron and workflow_dispatch are the safety nets.
• Result. Merging this PR updates https://pilotprotocol.network/install.sh within about a minute, plus up to 5 minutes of client or proxy cache.
• Fallback copy. The website fallback copy is updated in install.sh: sync the Pages fallback with pilot-protocol/release#49 website#263 (draft). Merge it right after this PR, otherwise the website sync run goes red on its drift step (the live URL is still correct, because R2 is written first).
• No CI here. This repo has no PR CI for install.sh, so the tests below were run by hand.

Changes

Root and transport

• Root is allowed on Linux without /run/systemd/system (containers and VMs such as Muse) when root itself runs the installer, and prints a note. Root is still refused, unless PILOT_ALLOW_ROOT=1:
  • on hosts with systemd, on macOS, and with launchd;
  • on any host when root comes from sudo, doas or pkexec for a regular user (SUDO_UID/SUDO_USER, DOAS_USER, PKEXEC_UID). curl … | sudo sh on WSL, OpenRC/runit hosts or dev containers would otherwise install into /root/.pilot, which that user cannot run, and sudo -E would leave a root-owned ~/.pilot. The error names the user and gives the command to run instead.
• --transport auto|udp|compat (or PILOT_TRANSPORT):
  • udp and compat are saved.
  • auto is never saved.
  • A saved transport=auto is rewritten to udp for a daemon that predates it.
  • Leaving compat restores the raw registry that an old-pilotctl compat install wrote.

No raw-IP endpoints behind a proxy

• A fresh config.json leaves out 34.71.57.205:9000 and 34.71.57.205:9001 when the node runs compat, or auto behind a proxy. The daemon then applies the endpoint that fits its transport: registry.pilotprotocol.network:443 over TLS in compat mode or through a proxy.
• With a pilotctl that predates --transport (v1.13.x forwards config.json's registry verbatim), a compat install gets registry.pilotprotocol.network:443 by name.
• The service units omit the stock endpoints in the same cases.
• A custom PILOT_REGISTRY or PILOT_BEACON is always kept.

Rotating credentials. Nothing credential-bearing is ever written or printed.

• proxy_cmd is saved in config.json: $PILOT_PROXY_CMD, or in a sandbox whose HTTPS_PROXY/https_proxy carries credentials, the credential-preferring command pilotctl uses:
  bash -c 'case $https_proxy in *@*) printf %s "$https_proxy";; *) printf %s "${HTTPS_PROXY:-$https_proxy}";; esac'
  • An existing proxy_cmd is never replaced.
  • It is also saved for a daemon that predates -proxy-cmd: that daemon ignores the key until it is upgraded, and the installer prints a note saying so.
• Downloads survive a rotation mid-install. pcurl retries once after re-reading the proxy URL from that command, mirroring the daemon's 407 → refresh → retry-once. The re-read URL only changes this process's environment. pilotctl skills check gets the refreshed credentials too.
• A PILOT_PROXY http(s) URL carries the downloads when no *_PROXY variable is set.
• The proxy is always shown redacted (http://***@host:port).
• The service-unit note says that saving a credentialed proxy stores the credentials, and offers proxy_cmd instead.

Honest output with today's releases (v1.13.9, v1.13.10: no -proxy)

• When a proxy is set but the installed daemon cannot use one, the installer warns in every transport. The warning points at step 3 of the pilot-sandbox recipe (https://pilotprotocol.network/learn/install-pilot-skills-in-meta-muse) rather than at a release that is not out yet.
• Where the proxy is the way out (a credential-bearing proxy on Linux without systemd, which is Muse, or PILOT_PROXY_CMD set), no output tells the agent to run pilotctl daemon start. That covers the warning, the no-systemd hint, GET STARTED step 0 and the re-run summary: each says not to run it and points at the recipe. On other hosts with such a proxy, the command stays, with the condition and the recipe next to it.
• auto is no longer announced before the download. The install and update summaries state the transport of the installed daemon. For v1.13.9 that is udp (… predates auto and never falls back to TCP 443 by itself), plus, when nothing chose udp, the --transport compat re-run for UDP-blocked hosts. compat saved for a daemon without -transport is reported as udp.

Release pins (pre-existing bug)

• --version <tag> and --channel beta aborted with "integrity anchors disagree" for every tag except latest_stable, because the manifest only hashes latest_stable. The manifest hash is now compared only for the tag it describes: its platform url names the tag, or the tag is latest_stable, which covers the managed-runtime manifest without urls. Any other tag is verified against checksums.txt. The tag the manifest describes still needs both anchors to agree.
• --help prints the usage header up to "WHAT THIS SCRIPT DOES" instead of a fixed line range.

Onboarding text

• Replies are read from the output of send-message --wait (JSON: .data.reply), not from "the newest inbox message".
• The appstore install examples drop the routine --force.
• An empty Email: line now explains the synthesized identity.
• The manual pointer names pilotctl skills instead of one harness path.

Safety

• The pilotctl daemon start --help probe now runs only for compat with a daemon that has -transport (v1.11+). Per-command help exists since v1.10, so the probe can never start a daemon. tests/managed-install.sh caught the unconditional version.

Round 2 review fixes (c36ad9b)

• R49-A2: sandbox proxy_cmd only for credentials a fresh shell can print. ENV_PROXY_CREDS is read before PILOT_PROXY is copied into https_proxy/HTTPS_PROXY for the downloads, and the sandbox command is chosen only when the proxy environment carries credentials and PILOT_PROXY is unset or auto (pilot-daemon runs a proxy command in place of the URL it would use, so it would have replaced an explicit PILOT_PROXY, as pilotctl already avoids). Before, PILOT_PROXY=http://u:p@… saved a command that printed nothing and the summary claimed "rotation needs no restart".
• R49-A1: restart advice stops the running daemon first. On a proxy-only host the "No managed service was running" advice now prints Stop the running daemon first: pilotctl daemon stop, then the recipe; the recipe starts a daemon but never stops one, and following the old advice ran two daemons with one identity.
• R49-A4: macOS. The LaunchAgent start line goes through start_hint like every other start line (condition on PROXY_UNSUPPORTED, "Do not run" on PROXY_ONLY), and nothing points macOS at the Linux-root-only recipe: PROXY_REMEDY there is "re-run this installer once a Pilot release whose pilot-daemon -h lists -proxy is out".
• R49-A5: --transport auto with a pre-auto daemon no longer says "keeps its default (udp)" while a saved compat stays in effect; it says the saved transport is kept and how to switch.
• R49-A3 (pilot-mcp#23 did not recognise this installer's proxy_cmd): fixed in pilot-mcp#23 (its SANDBOX_PROXY_CMD is now this exact command; the older pre-release one is still recognised).

Tests for round 2 (tests/proxy-transport-install.sh, new darwin fixture with a fake launchctl): 4d (PILOT_PROXY credentials: no proxy_cmd, no rotation claim, no leak; explicit PILOT_PROXY next to a credentialed HTTPS_PROXY: no proxy_cmd), the re-run advice keeps pilotctl daemon stop, 3b (old daemon, --transport compat then --transport auto: compat stays and is reported), 10a (macOS with a proxy: conditioned LaunchAgent line, no Linux recipe; with PILOT_PROXY_CMD: "Do not run launchctl load -w …"). Passes on macOS sh and in a Linux container as root, as a regular user and with /run/systemd/system; tests/managed-install.sh passes; shellcheck -s sh install.sh tests/*.sh clean.

E2E (round 3 Muse sim, rotating-credential CONNECT-443 proxy, root, no systemd, packet sink for direct dials): installed through pilot-skills#34's one-shot with PILOT_INSTALL_URL=file:// this c36ad9b script — with the next release's binaries: rc=0, config.json transport=compat + proxy_cmd (the pilotctl command), node registered through the proxy; with released v1.13.10 and root without CAP_SYS_ADMIN: the WARNING, the no-systemd hint and GET STARTED name the recipe and say not to run pilotctl daemon start (no output line tells the agent to run it).

Tests

Fixture tests

• sh tests/proxy-transport-install.sh: OK under macOS sh and dash. It uses a fixture release with fake curl, uname, id and sudo, and needs no network. New cases:
  • a credential rotation mid-install: the fake curl returns 407 once the password rotates, and BASH_ENV stands in for Muse's fresh-shell credentials;
  • no raw-IP endpoint for auto or compat behind a proxy;
  • compat with an old pilotctl;
  • downloads through PILOT_PROXY;
  • old daemon: proxy_cmd saved, plus the warning with the recipe link;
  • root refused on macOS;
  • root refused under sudo (SUDO_UID+SUDO_USER, SUDO_USER alone) and doas. It is allowed under sudo from root (SUDO_UID=0) and with PILOT_ALLOW_ROOT=1;
  • v1.13.x in a proxy-only sandbox, on the first install, a re-run and --transport compat: no line tells the agent to run pilotctl daemon start (or stop && start, or systemctl enable), and the warning, the no-systemd hint and GET STARTED all point at the recipe. With a proxy that has no credentials, the command stays with the condition. A daemon that can use the proxy keeps the command and gets no recipe;
  • v1.13.x without a proxy: Transport: udp (… predates auto, never Transport: auto, plus the compat hint. There is no hint after --transport udp or with a daemon that has auto;
  • a daemon without -transport and --transport compat: the summary says udp;
  • --version v9.9.8 --yes and --channel beta against a manifest that hashes only latest_stable, with and without platform urls: installed and verified by checksums.txt. The latest_stable tag with a disagreeing manifest hash, and a pinned tag that the manifest's url names, are still refused;
  • the --help range;
  • no credentials in the output or anywhere under ~/.pilot.
  • PILOT_TEST_SH=dash dash tests/proxy-transport-install.sh runs the installer itself under dash.
  • Mutation check: disabling the refresh, the endpoint omission, or the PILOT_PROXY export each fails the suite. So does each review fix disabled on its own: the sudo refusal, the manifest-tag rule (either half), proxy-only detection, GET STARTED step 0, the compat hint, or restoring the early auto banner.
• sh tests/managed-install.sh: OK.
• shellcheck -s sh install.sh tests/*.sh: clean. sh -n passes under busybox, dash and bash.

Docker, review fixes (head 67e83fc, real v1.13.9 downloads)

Check	Result
ubuntu:24.04, no systemd, user agent with NOPASSWD sudo: cat install.sh | sudo sh and sudo -E sh install.sh	Both rc=1: refusing to install as root: this runs under sudo for agent. No /root/.pilot, no ~/.pilot, no /usr/local/bin links. Base 36c38dc refuses too; the previous head installed into /root/.pilot.
Same container, root directly (HOME=/root, no SUDO_*)	rc=0 with the sandbox note; links point to /root/.pilot/bin.
Same container as agent, plain install	rc=0. No auto banner. Summary: Transport: udp (this pilot-daemon, v1.13.9, predates auto …) plus the --transport compat re-run. That re-run saves compat with registry.pilotprotocol.network:443 and the update summary says compat.
--version v1.13.8 --yes, --channel beta (v1.13.10-rc.2), --version v1.10.1 --transport compat --yes	All rc=0, Verified SHA-256 (checksums.txt), and pilotctl version matches. Base 36c38dc: --version v1.13.8 gives integrity anchors disagree.
Muse rig: internal network, CONNECT-443-only authenticating proxy that rotates its password every 5s, BASH_ENV fresh credentials, poisoned DNS, root HOME=/root, no systemd. Fresh install, re-run, and --transport compat	All rc=0. For each: 0 lines that tell the agent to run pilotctl daemon start, 3 recipe pointers (warning, no-systemd hint, GET STARTED), 0 Transport: auto, 0 credential matches in the output, /root or /usr/local/bin. Three 407s from rotations were each retried and allowed. The proxy log has only :443 CONNECTs.

Docker, served path

curl -fsSL https://raw.githubusercontent.com/pilot-protocol/release/<head>/install.sh | sh in a Muse-like rig with:

• an --internal network;
• an authenticating CONNECT-443-only proxy that reads its password per request;
• the password rotated every 5–6s;
• BASH_ENV exporting the current credentials;
• the Pilot hosts poisoned to 198.18.67.197;
• root with HOME=/root and no systemd.

Scenario	Result
A: --transport compat, v1.13.9, 5 runs	rc=0 in 7–9s each. Rotations hit 407 on github.com, release-assets.githubusercontent.com and pilotprotocol.network, and every one was retried and allowed. The proxy log has only :443 CONNECTs; direct egress fails (Could not resolve host). config.json = {transport: compat, registry: registry.pilotprotocol.network:443, proxy_cmd: <command>}, no raw IP. 0 credential matches in the output, /root or /usr/local/bin. The v1.13.9 warning names the sandbox recipe.
A2: plain curl | sh, v1.13.9, 3 runs	rc=0. v1.13.9 has no auto, so the transport is udp and the stock endpoints stay, which is correct for that daemon. proxy_cmd is saved and the warning is shown.
B: same rig, plain install, with the next release's binaries	Binaries built from pilotprotocol feat/native-https-proxy @ 5097bc22 and copied over the download by a test-only copy of this script. Output: Transport: auto. config.json has no registry or beacon and has proxy_cmd. Then pilotctl daemon start registered in 5s (transport auto-selected transport=compat, registry=registry.pilotprotocol.network:443, node 251998, since deregistered). Across 3+ rotations (30s period): 4 registry reconnects got 407 and were each retried and allowed. pilotctl --json send-message list-agents --data '/data {"search":"weather","limit":2}' --wait returned .data.reply, and pilotctl appstore catalogue worked. 0 credential matches in the daemon log.
B, compat / re-run / --transport udp	Compat: no registry key. A re-run keeps the config. --transport udp saves udp.
C: normal network, non-root (agent), no proxy	rc=0. config.json has the stock registry and beacon as before. No root note.
C, root	Root with /run/systemd/system present is refused (refusing to install as root, no ~/.pilot). Root without systemd is allowed with the note.

Deploy order (all six PRs)

1. installer: transport auto/compat, HTTPS-proxy sandboxes, root without systemd #49 merge → notify-install-sh-sync dispatches to pilot-protocol/website → sync-install-sh.yml puts install.sh into R2 pilot-release-assets/install.sh (the pilot-release Worker reads R2 per request; no Worker deploy). Check: curl -fsSL https://pilotprotocol.network/install.sh | shasum -a 256 → 4384f0cb2890177cb87a2e2bc49e1b814d2850434d24c54d1730c9955fbbb671 (up to 5 min cache).
2. install.sh: sync the Pages fallback with pilot-protocol/release#49 website#263 merge (Pages fallback public/install.sh, same bytes) → the sync workflow's drift step goes green.
3. feat: native HTTPS-proxy support and -transport=auto (Meta Muse / proxy-only sandboxes) pilotprotocol#470: re-run install.sh matches pilot-protocol/release, merge.
4. Tag pilotprotocol v1.13.11 on main (release workflow: binaries, checksums.txt, signed latest.json). v1.13.10 shipped without #470, so until this tag every install gets a daemon without -proxy. Check pilot-daemon -h lists -proxy and -proxy-cmd.
5. feat(muse): one-shot Pilot-in-Muse installer and pilot-up.sh TeoSlayer/pilot-skills#34 merge → CI regenerates skills.json/setups.json and skills/pilotctl/SKILL.md, dispatches the website deploy; the Muse one-shot (raw.githubusercontent.com/TeoSlayer/pilot-skills/main/muse/install.sh) is live at merge.
6. feat(setup): install and start Pilot behind an HTTPS egress proxy (Meta Muse) pilot-mcp#23 merge → release 0.3.0 (behaviour change; last published 0.2.13): bump package.json, package-lock.json, server.json (both fields), .well-known/mcp/server-card.json, src/version.js, src/openclaw-plugin/openclaw.plugin.json, src/openclaw-plugin/evaluate.js and the pinned pilotprotocol-mcp@0.2.13 strings in test/{hermes-setup,openclaw-plugin,native-harness-setup,harness-config-contracts}.test.js (test/release-contract.test.js locks them), move [Unreleased] to [0.3.0], push tag v0.3.0 → publish.yml (tag must equal v+version; lint + tests; npm via OIDC, MCP Registry server.json, ghcr.io/pilot-protocol/pilot-mcp).
7. Add marker-gated skill targets (gatedTools) and the Meta Muse format skillinject#42 merge → tag v0.2.5 → pilotprotocol follow-up PR go get github.com/pilot-protocol/skillinject@v0.2.5 && go mod tidy (GOWORK=off) → next daemon release (v1.13.12; or fold the bump in before tagging v1.13.11).
8. inject-manifest: gate Meta Muse under a new gatedTools key TeoSlayer/pilot-skills#35 merge (manifest gatedTools muse row). Released daemons drop the key (CI released-skillinject v0.2.2–v0.2.4). No manifest signing step: DefaultManifestPublicKeyHex is empty; only hosts that installed ~/.pilot/skillinject.pub verify inject-manifest.json.sig.

🤖 Generated with Claude Code