Skip to content

feat(mobile): preview agent devices with scoped stream access - #832

Merged
rynfar merged 4 commits into
pylonfrom
upstream/2026-09-24-mobile-agent-devices
Sep 27, 2026
Merged

rynfar merged 4 commits into
pylonfrom
upstream/2026-09-24-mobile-agent-devices

Conversation

@rynfar

@rynfar rynfar commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator

What changed

Mobile now shows device sessions for the current thread and opens a full-screen viewer with live video and controls. The viewer shares Pylon's device stream transport with web, keeps host identity when switching devices, and stops decoding when the viewer loses focus, the app backgrounds, or the route closes.

Simulator testing exposed two release-blocking bugs in the first draft: the live subscription's waiting flag kept the preview button hidden even after a valid session arrived, and the iOS full-screen modal hid its Close, Home, and device menu controls. Device inventory is now tied to the exact RPC session and supervisor generation; the iOS header is visible from the modal's initial render. Stream tickets remain fenced to the prepared connection and generation, and neither React keys nor errors contain ticket material.

Adapted from upstream sources 823119350ee29cca434b1295aa5c560f4c9ef98a and the shared/mobile recovery portions of 9f73ca3677; Pylon's existing web recovery remains in place.

iOS evidence

Before the final fix, the viewer streamed the device but had no native controls:

iOS device viewer before the header fix

After the fix, an iPhone 17 showed a live iPhone 16e Settings screen with Close, device options, and Home controls:

iOS device viewer with live stream and controls

Validation

  • 90 focused mobile, shared-runtime, and web device tests passed; mobile, web, and client-runtime typechecks passed.
  • Scoped lint, format, and diff checks passed. The generated mobile WebView stream bundle built successfully.
  • Isolated iOS Simulator verification used a synthetic thread and separate viewer/target simulators. The viewer updated when the target opened Settings; Home moved the target to the iOS home screen. The device menu opened, and Close returned to the thread.
  • Independent adversarial review of the reconciled feature and the simulator-driven fix found no concrete issue.

Limits

Android native rendering was not tested because this host has no Android SDK or emulator. Android has an explicit decoder error when WebCodecs is unavailable; remote Android streaming needs a secure context such as HTTPS or loopback. Older servers without device-state and hub endpoints cannot provide this viewer; the existing mobile thread UI remains usable.

Model/harness: GPT-6 Codex implementation; GPT-6 Sol (low) independent review.

@vercel

vercel Bot commented Sep 24, 2026

Copy link
Copy Markdown

Deployment failed for project pylon-marketing with the following error:

Resource is limited - try again in 24 hours (more than 100, code: "api-deployments-free-per-day").

Learn More: https://vercel.com/pylon-code?upgradeToPro=build-rate-limit

@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XXL labels Sep 24, 2026
@rynfar rynfar changed the title Mobile: preview agent devices with scoped stream access feat(mobile): preview agent devices with scoped stream access Sep 24, 2026
@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 14.0 KiB 14.0 KiB +12 B (+0.1%) 15.1 KiB ✅
Codex Thread snapshot wire 7.3 KiB 7.3 KiB −3 B (−0.0%) 7.3 KiB ✅
Codex Live turn WebSocket wire 6.7 KiB 6.7 KiB +15 B (+0.2%) 7.8 KiB ✅
Codex Live turn WebSocket decoded 58.0 KiB 58.0 KiB +44 B (+0.1%) 66.4 KiB ✅
Codex Live turn messages 9 10 +1 (+11.1%) 21 ✅
Claude Total thread wire 14.0 KiB 14.0 KiB +7 B (+0.0%) 15.1 KiB ✅
Claude Thread snapshot wire 7.3 KiB 7.2 KiB −9 B (−0.1%) 7.3 KiB ✅
Claude Live turn WebSocket wire 6.7 KiB 6.8 KiB +16 B (+0.2%) 7.8 KiB ✅
Claude Live turn WebSocket decoded 58.9 KiB 58.9 KiB 0 B (0.0%) 66.4 KiB ✅
Claude Live turn messages 9 9 0 (0.0%) 21 ✅

Baseline: c05418c · PR result: 7ebae69 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 115.7 KiB
  • Claude decoded thread snapshot: 116.4 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

@vercel

vercel Bot commented Sep 27, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
pylon-marketing Ignored Ignored Preview Sep 27, 2026 12:05am UTC

@rynfar
rynfar marked this pull request as ready for review September 27, 2026 00:15
@rynfar
rynfar merged commit dfe598d into pylon Sep 27, 2026
22 checks passed
@rynfar
rynfar deleted the upstream/2026-09-24-mobile-agent-devices branch September 27, 2026 00:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant