fix(media): preview host files and stream videos across clients

[juliusmarminge]

What changed

Agent-referenced images and videos outside the workspace could not be previewed, selecting MP4s in Files tried to read them as text, and web attachment previews downloaded entire videos into blobs.

• Preview supported media from Markdown links, reference links, recognized inline-code paths, image/video embeds, and viewed-image work logs. Absolute paths refer to the environment host; relative paths resolve against the thread workspace or worktree.

• Show paused video previews with an initial frame, native controls, and expansion on web and mobile. Web chat embeds and file viewers share MediaVideoPlayer; the other web previews share first-frame preparation. Mobile reuses the bounded native-thumbnail cache and extraction queue. Inline playback begins only after Play, while opened viewers load without autoplay.

• Use the same filmstrip artwork for supported video formats in web and mobile file chips and file lists. Extension detection follows the shared video MIME map.

• Play videos in both file viewers without text reads or text preloads. Web video attachments pass signed URLs directly to the player so the existing Range/206 route handles buffering. Workspace refreshes wait for active playback to pause; ordinary signed-URL renewal preserves paused position.

• Authorize local media with expiring signed URLs bound to the canonical path and device/inode identity. Validate a regular media file, then stream from the same guarded descriptor. Refresh descriptor metadata before setting response lengths and ranges. Do not copy previews into attachment storage. Removed, moved, or atomically replaced files invalidate old URLs. Mutable host videos omit cache validators and use private, no-store, while ordinary byte ranges still return 206.

• Render the shared web viewer through a body portal so clipped Markdown containers cannot contain it. Failed direct HTTP(S) and protocol-relative previews retain Open original navigation; generated host-media URLs are not exposed as original links. Protocol-relative media links open the viewer on both clients, with native clients explicitly using HTTPS.

• Retry, workspace refresh, and mobile Save/share request fresh capabilities. Signed-URL renewal does not reset a loaded player's position.

• Keep existing workspace HTML/PDF/SVG restrictions and explicit-download behavior. Enable streaming through the Electron protocol.

• Preserve original source references in shared media actions across inline previews, expanded previews, and file viewers. Web images support save/copy; video downloads use browser controls. Mobile reuses native sharing.

Markdown source → rendered UI

These captures use a synthetic thread and files on the environment host, including files outside its workspace.

Image and video embeds

Raw Markdown source → rendered UI below

## Media outside the workspace

![Outside-workspace image](/tmp/t3-media-verification-2hud6jgq/outside-media/outside-image.png)

![Outside-workspace video](/tmp/t3-media-verification-2hud6jgq/outside-media/outside-video.mp4)

Both previews use the original files. Video playback starts only when requested.

Web · Chromium	Mobile · Android

Both screenshots show paused previews. The recordings show playback followed by expansion into a paused viewer.

Recordings: web 28s and Android 22s

Web · Chromium	Mobile · Android
Play web recording · 28s	Play Android recording · 22s

Before: empty web video preview

With preload disabled, no frame appeared until Play:

Expanded, paused video viewers

Web · Chromium	Mobile · Android

Links, reference links, and inline-code paths

Raw Markdown source → rendered UI below

## Open media references

- [Image link](/tmp/t3-media-verification-2hud6jgq/outside-media/outside-image.png)
- [Video link](/tmp/t3-media-verification-2hud6jgq/outside-media/outside-video.mp4)
- [Reference video link][recording]
- Image path: `/tmp/t3-media-verification-2hud6jgq/outside-media/outside-image.png`
- Video path: `/tmp/t3-media-verification-2hud6jgq/outside-media/outside-video.mp4`
- [File URL image](file:///tmp/t3-media-verification-2hud6jgq/outside-media/outside-image.png)
- [Missing image](/tmp/t3-media-verification-2hud6jgq/outside-media/missing-image.png)

[recording]: /tmp/t3-media-verification-2hud6jgq/outside-media/outside-video.mp4

### Workspace-relative image

![Workspace image](media/workspace-image.png)

### Code stays code

```text
Recorded evidence here: /tmp/t3-media-verification-2hud6jgq/outside-media/outside-video.mp4
```

Web · Chromium	Mobile · Android

Before: video chips used a generic file icon

Opening an image reference and reporting a removed file

Web · Chromium	Mobile · Android

Web · Chromium	Mobile · Android

Remote previews that are actually web pages

The following Markdown file intentionally links to HTML responses with media-looking paths. The shared web viewer covers the whole window and offers Open original when decoding fails. It never navigates automatically.

Raw Markdown source → rendered UI below

# Remote preview fallback

These links intentionally return an HTML page, not image or video bytes.

[Image page](http://localhost:8057/not-an-image.png)

[Video page](http://localhost:8057/not-a-video.mp4)

Image page	Video page

File viewers: paused frames and streaming playback

Before: reported web file-viewer failure

This is the reported failure for an MP4 opened in the web file viewer. The after captures use separate synthetic fixtures.

Web · Chromium	Mobile · Android

Web opens a 90-second, 45 MB test-pattern video. Android opens the five-second media/workspace-video.mp4; its reused artwork says "Outside-workspace video", but this file is inside the workspace.

Recordings: web 31s and Android 25s

Web · Chromium	Mobile · Android
Play web recording · 31s	Play Android recording · 25s

The Chromium pass showed a paused initial frame at 0.1 seconds with only 3.01 seconds buffered from the 90-second video. Metadata/frame loading used Range: bytes=0- and a later Range: bytes=458752-, both answered with 206 Partial Content. Playback and a seek to the middle worked through the same native player. The app does not fetch a playback blob.

Preview loading now deliberately reads video data before Play. Web preloads metadata only near the viewport for inline videos; mobile gates native thumbnail extraction on visible chat rows. Preload is a browser hint, not a strict byte limit. No server-generated poster files or imported video copies are needed.

Review regression evidence

Authored dimensions, unavailable media, and protocol-relative links

The deliberately unavailable URLs return HTML to exercise decode failures. Retry keeps the fallback usable, Open original retains the authored URL, and image/video dialogs share the same error card. The sized video remains paused at its initial frame.

Raw Markdown source → rendered UI below

# Video preview recovery

## Authored dimensions

<img src="/tmp/t3-media-verification-2hud6jgq/outside-media/outside-video.mp4" alt="Sized video" width="320" height="180">

## Remote video unavailable

![Remote video](http://localhost:8057/not-a-video.mp4)

## Protocol-relative video unavailable

![Protocol-relative video](//localhost:8057/not-a-video.mp4)

Both unavailable examples intentionally return HTML. Retry or open the original page.

## Protocol-relative media link

[Open protocol-relative video](//localhost:8057/not-a-video.mp4)

## Remote image unavailable

![Remote image](http://localhost:8057/not-an-image.png)

Image and video error dialogs

Image	Video

Android file viewer, playback, and explicit sharing

The latest Android pass opens a video in Files, expands the paused preview, plays it to the end, and invokes Save or share video. Only that explicit share operation downloads a client-side copy. No share target was selected. Returning from the share sheet preserves the final frame without a loading spinner.

File preview	Native share sheet

Play Android file viewer, playback, and sharing recording · 29s · Paused expanded preview · Ended state after sharing

Workspace links open the file viewer

Workspace image and video chips now open the normal file viewer on web and mobile. Out-of-workspace media links and the expand controls on inline embeds keep their media previews. The demo now labels deliberately absent files as expected failures.

Raw Markdown source

[Open the larger video](media/large-streaming-check.mp4)

Before: workspace chip opened the expanded modal	After: workspace chip opens a file-viewer tab

Watch the web link-to-file-viewer recording

Watch the Android link-to-file-viewer recording

Verified the 90-second video plays in both clients. The web file viewer uses the signed HTTP streaming URL, not a blob; a range request returned 206 with Content-Range: bytes 1048576-1048703/45066088. The routing follow-up passed 158 existing focused tests, web/mobile typechecks, and scoped lint and formatting. No new test dependencies or DOM harnesses were added.

Source paths and media actions

Previews retain the authored path or URL separately from their playback URL. Web hover tooltips show that reference, and right-click menus copy it. Workspace media can copy its relative path or open the file viewer. Images can be saved or copied to the clipboard. Mobile uses long-press and overflow menus with native Save or share.

Raw Markdown source for the previews below

![Outside-workspace image](/tmp/t3-media-verification-2hud6jgq/outside-media/outside-image.png)
![Outside-workspace video](/tmp/t3-media-verification-2hud6jgq/outside-media/outside-video.mp4)

![Workspace image](media/workspace-image.png)
![Workspace video](media/workspace-video.mp4)

Web right-click	Android long-press

Web hover tooltip showing the original path

Before: previews did not expose source actions

Expanded previews and file viewers

Web expanded preview	Android expanded preview

Web file viewer	Android file viewer

Watch web source actions, clipboard paste, expansion, and file-viewer navigation · 45s · Watch Android expanded actions and file-viewer navigation · 18s · Android native video share sheet

Video downloads use the browser

Removed the web context menu's fetch-based Save video action. The native player keeps its Download control; source-copy and file-viewer actions remain in the context menu. No application-side video fetch is needed for downloading. An undecodable video offers Open in browser, while remote failures retain Open original. Existing local blob attachments use a native download link.

Raw Markdown source for the external video

![Embedded web recording](https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/3df4ab52b599357e/web-chat-first-frame.mp4)

Source action	Browser's native controls

Failed video recovery

The intentionally invalid MP4 exercises decode failure in the file viewer. The browser link opens the original streamed bytes without a fetch-and-blob download.

Before	After

Verified actual clipboard paste for the full host path, authored external URL, and copied image. The saved image matches the original SHA-256. Android shared the original filename through the native share sheet, and expanded previews returned to the file viewer. Source actions never copy generated playback capabilities. Empty file-viewer padding no longer opens the image menu. Focused tests, scoped typechecks, lint, and formatting passed; no UI/DOM harnesses were added.

Verification

• Focused tests passed after removing the DOM/mock-native harnesses and presentation/wiring checks. No new test dependencies remain.
• Scoped web, mobile, and server typechecks were rerun. Shared, desktop, contracts, and client-runtime checks passed during the earlier PR verification.
• Targeted lint, formatting, and git diff --check passed.
• Verified host-video headers and playback in Chromium after the conditional-range fix. A bytes=0-31 request returned 206 with exactly 32 bytes; unverifiable If-Range correctly returned 200.
• Actual Chromium and Android emulator passes covered chat previews, filmstrip icons, playback, expansion, missing files, and MP4 file viewers. Chromium additionally covered failed remote image/video links and the body-level dialog portal. Host-media previews did not populate attachment storage.
• Kept regression coverage for canonical file identity and symlink races, in-place edits and replacement, range bounds and descriptor cleanup, filename-versus-URL decoding, and first-frame seek guards. UI behavior is covered by the recorded Chromium and Android passes above.
• iOS and Electron were not manually exercised. No repo-wide checks were run. Evidence is hosted externally; no PR-only screenshots or recordings are committed.

Follow-ups from the rendering audit

• Replace the legacy Android attachment modal's download-before-playback path. The new Android Markdown and file-viewer paths stream; the legacy attachment path is unchanged.
• Reuse chat media resolution/rendering in mobile Markdown-file previews. This PR adds playback for selected video files, not media embeds inside those Markdown files.

Built with Codex in T3 Code.

---

Note

High Risk
Changes signed asset authorization, symlink-safe file open, and video HTTP semantics (security-sensitive), plus broad client routing from markdown/workspace media to new preview flows.

Overview
Adds end-to-end host media preview so images and videos referenced in chat, workspace files, and markdown can play without reading MP4s as text or buffering whole files into blobs.

Server introduces a media-file asset with signed URLs tied to canonical path plus device/inode identity. Responses stream from a guarded descriptor with byte-range (206) support, private, no-store for videos, and invalidation when files move or are replaced atomically.

Mobile gains inline and full-screen MediaVideoPlayer, shared useMediaActions (copy path/URL, open file viewer, native share), workspace video preview in Files, and richer markdown handling (protocol-relative URLs, UNC file: paths, path-shaped inline code, video file icons). Asset URLs can be refreshed on retry/share without resetting playback position; list viewability gates thumbnail work.

Desktop widens CSP media-src to http:/https: and enables stream: true on the custom scheme for streaming playback.

Web adds useAssetUrlRefresh for re-minting capabilities after file changes (aligned with mobile).

^{Reviewed by Cursor Bugbot for commit 0977bfc. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Add media-file asset type with signed streaming URLs and video preview across web, mobile, and desktop

• Introduces a media-file asset resource in packages/contracts/src/assets.ts; the server issues identity-bound tokens (device/inode) and serves guarded byte-range streams via openMediaFile in assetFileResponse.
• Adds full-screen and inline video players (MediaVideoPlayer on web, MediaVideoPlayer on mobile) with URL re-signing on retry, viewport-aware preloading, and expand-to-modal support.
• Updates markdown rendering on both clients to classify and render videos inline from workspace paths and remote URLs, with media action menus (copy path/URL, share, open file) via MediaActions and useMediaActions.
• Replaces blob-based video download with signed URL streaming in ChatView.tsx and ExpandedImageDialog.tsx; removes loadVideoPreviewUrl and downloadVideoPreview.
• Fixes protocol-relative URL handling in resolveExternalWebLinkHost, normalizeNativeMarkdownUrl, and resolveMarkdownFileLinkTarget.
• Desktop CSP in ElectronProtocol.ts now allows http:/https: in media-src and enables stream: true for the custom scheme.
• Risk: asset tokens for media-file bind to device/inode identity; if a file is replaced or truncated in place, resolveAsset rejects the token and the client must call useRefreshAssetUrl/useAssetUrlRefresh to re-sign. The old workspace-file resource tag for images is replaced by media-file across presentation.ts and test expectations, which may break out-of-tree consumers expecting _tag: "workspace-file".

^{Macroscope summarized 0977bfc.}

[github-actions]

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	13.3 KiB	13.3 KiB	+7 B (+0.1%)	15.1 KiB	✅
Codex	Thread snapshot wire	6.9 KiB	6.9 KiB	−5 B (−0.1%)	7.3 KiB	✅
Codex	Live turn WebSocket wire	6.4 KiB	6.4 KiB	+12 B (+0.2%)	7.8 KiB	✅
Codex	Live turn WebSocket decoded	55.6 KiB	55.6 KiB	0 B (0.0%)	66.4 KiB	✅
Codex	Live turn messages	10	10	0 (0.0%)	21	✅
Claude	Total thread wire	13.3 KiB	13.3 KiB	−53 B (−0.4%)	15.1 KiB	✅
Claude	Thread snapshot wire	6.9 KiB	6.9 KiB	−3 B (−0.0%)	7.3 KiB	✅
Claude	Live turn WebSocket wire	6.4 KiB	6.4 KiB	−50 B (−0.8%)	7.8 KiB	✅
Claude	Live turn WebSocket decoded	56.4 KiB	56.3 KiB	−88 B (−0.2%)	66.4 KiB	✅
Claude	Live turn messages	10	8	−2 (−20.0%)	21	✅

Baseline: 60cef47 · PR result: 0977bfc · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

• Codex decoded thread snapshot: 109.4 KiB
• Claude decoded thread snapshot: 110.1 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

[macroscopeapp]

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR adds a substantial cross-platform media-preview capability and changes host-file authorization, signed asset serving, HTTP streaming, and default file-preview behavior. Its broad runtime and sensitive file-access scope, together with new static-analysis suppressions, warrants human review.

^{You can add or adjust custom eligibility rules. Learn more.}

[juliusmarminge]

@macroscope Please rerun the UI Consistency check on d84a5319081457abf8f42fa69e6bb2a017df32e4 after considering the test-scope decision. The sole finding is acknowledged and resolved as an intentional maintainer-requested coverage tradeoff, not a UI defect. No check configuration has been changed.

The PR now includes fresh before/after screenshots and Chromium/Android recordings of workspace chips opening the file viewer, with the raw Markdown source beside the rendered evidence. CI tests and typechecks pass; real Chromium playback uses the signed HTTP URL and returns 206 for byte ranges.

[macroscopeapp]

Manual reviews triggered for commit d84a531:

• Approvability Check
• Correctness Check

• Effect Service Conventions
• UI Consistency

^{All prior checks · these links stay valid even if you push more commits.}

[macroscopeapp]

Just FYI for future @mentions, I'm Macroscope-App, not Macroscope.

Rerun triggered. Results will be posted as check runs.