Skip to content

Cloud Disconnect: one-click unconfirmed header button fully deregisters the tower — likely cause of repeated cloud drops during uplink flaps #1370

Description

@waleedkadous

Symptom

Tower dropped its cloud.codevos.ai registration twice today (2026-08-08 18:28:03Z and 20:25:41Z), each requiring a fresh OAuth. The owner experienced this as "the tower keeps disconnecting from cloud — it's been robust until now."

What the log shows (both incidents, identical signature)

Tunnel: connected → disconnected          ← + ~5 rapid connect/disconnect cycles (~60ms apart)
Server-side deregister succeeded for tower <id>
Cloud config removed or invalid, disconnecting tunnel

This is the POST /api/tunnel/disconnect handler signature (tower-tunnel.ts ~443–490): tunnel teardown → server-side DELETE → deleteCloudConfig() → config-watcher notice. Not a network failure and not key revocation:

  • The tunnel client treats invalid_api_key/auth_failed as log-and-advise only — it never deregisters (tunnel-client.ts:429).
  • The CLI afx tower disconnect deletes the local config before signaling Tower, so the Tower-side handler would find no config and could not log "Server-side deregister succeeded". Both incidents show the config still present when the handler ran → a direct POST.
  • Cron sweep across all registered workspaces: no task invokes tower/tunnel commands.
  • Active sibling agent sessions in other workspaces were polled and deny issuing it.

The hazard

The only production caller of that POST in the codebase is the dashboard header's cloud control (apps/web/src/components/CloudStatus.tsx:117-125):

  • One click, no confirmation, adjacent to the "Open" link.
  • In the disconnected state, a Connect button renders in the same header position.

Independent evidence (a sibling workspace's monitors + a cron comment dated 2026-08-04) shows the host uplink has been flapping for days, including a documented outage window overlapping the 20:25Z incident. During a flap the header control flips between Connect and Disconnect under the cursor; a click aimed at Connect/Open can land on Disconnect — and a single unconfirmed click deregisters the tower server-side and deletes local credentials.

Proposed fixes

  1. Confirmation dialog on the dashboard Disconnect button (it is a destructive, credential-deleting action; cf. how destructive actions are gated elsewhere).
  2. Source attribution logging on POST /api/tunnel/connect|disconnect (remote address, user-agent, and whether the request arrived via the tunnel proxy) so the next incident is attributable from the log alone.
  3. Defense in depth: have Tower reject tunnel-proxied requests to /api/tunnel/* management endpoints if that path isn't already blocked — a cloud-side actor should not be able to deregister the tower through its own tunnel.
  4. Optional: debounce/disable the header cloud control while the tunnel state is actively flapping (state changed < 2s ago) to remove the click-target swap hazard.

Activity

  1. waleedkadous commented on Aug 8, 2026

    @waleedkadous
    ContributorAuthor

    Update: owner attests no dashboard interaction at either deregistration time — the one-click-button theory is weakened for these two incidents (the UX hazard stands regardless). This elevates asks 2 and 3 (source-attribution logging; audit whether tunnel-proxied requests can reach /api/tunnel/*). Also worth checking: the isCloudHosted guard hiding the cloud controls is new — a stale cached dashboard bundle on a remote device accessing via codevos.ai would still render a functional Disconnect button. Companion stays-down defect filed separately (tunnel client wedges after uplink flaps and never self-recovers).

  2. waleedkadous commented on Aug 8, 2026

    @waleedkadous
    ContributorAuthor

    Correction from the builder (verified): CloudStatus.tsx is NOT wired into any UI (zero importers — dead code since spec 579). The live callers of POST /api/tunnel/disconnect are (1) the Tower homepage header in templates/tower.html (weak confirm()), and (2) the VS Code command "Codev: Disconnect Tunnel" (codev.disconnectTunnel → extension.ts:1307) with no confirmation — one fuzzy command-palette Enter away, and now the strongest accidental-trigger candidate for both incidents. Fix covers all three call sites + attribution logging.

  3. waleedkadous commented on Aug 8, 2026

    @waleedkadous
    ContributorAuthor

    Incident #3, 2026-08-08 22:29:15Z — laptop lid closed, user input impossible. Same signature (reconnect storm → server-side deregister → config deleted). Static analysis has now exonerated every known caller: tower.html (confirm-gated), VS Code command (palette-only), dashboard CloudStatus (dead code), CLI (different log signature), crons (none), sibling agents (denied + no mechanism). All three incidents immediately follow a sub-second reconnect storm, suggesting the deregister is reactive to connection churn. Prime theory now: a POST arriving through the tunnel's h2 session into Tower's API. Builder has been asked to prioritize source-attribution logging (with tunnel-vs-local origin) and the tunnel-origin rejection, plus audit whether tunnel-borne requests are auth-gated at the cloud edge. Until the fix lands, every reconnect is expected to survive only until the next uplink flap.

  4. added a commit that references this issue on Aug 9, 2026
  5. added a commit that references this issue on Aug 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/cross-cuttingTouches multiple areas — needs coordinated handling

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions