Skip to content

fix(cli): spec-compliant terminal image rendering + Windows clipboard… - #974

Open
smarpitm wants to merge 1 commit into
CodebuffAI:mainfrom
smarpitm:fix/terminal-image-rendering
Open

fix(cli): spec-compliant terminal image rendering + Windows clipboard…#974
smarpitm wants to merge 1 commit into
CodebuffAI:mainfrom
smarpitm:fix/terminal-image-rendering

Conversation

@smarpitm

@smarpitm smarpitm commented Aug 11, 2026

Copy link
Copy Markdown

What

Fixes terminal inline image rendering so attached photos display correctly in kitty-capable terminals, and hardens Windows clipboard image paste.

Why

Photos attached via /image or Ctrl+V could render "distorted":

  1. Kitty graphics protocol spec violations — the final chunk omitted the required m=0 terminator and every chunk repeated a=T/full control data, so terminals treated each chunk as a new image (fragmented display). The format id was also hardcoded to f=100 (PNG) even for the JPEG payloads the compression step produces.
  2. iTerm2 size param sent the base64 length instead of the decoded byte length (minus padding).
  3. Detection missed WezTerm / Ghostty / Warp / Konsole, which all support kitty graphics.
  4. Windows paste failed silently on machines without powershell.exe (no pwsh fallback).

Changes

  • cli/src/utils/terminal-images.ts — spec-compliant kitty chunking (m=0 on the last chunk, control data only on the first), format id derived from the real media type, iTerm2 size = decoded bytes, detection for WezTerm/Ghostty/Warp/Konsole (case-insensitive).
  • cli/src/utils/clipboard-image.tspwsh fallback when powershell.exe is missing, normalized string output.
  • cli/src/components/blocks/image-block.tsx, image-card.tsx — pass mediaType to the renderer.
  • New tests: terminal-images.test.ts (sequence conformance against the kitty spec), image-pipeline-integrity.test.ts (verifies image processing never corrupts payloads).
  • test/setup-scm-loader.ts — no-op stub so bun test in the cli package works in the public mirror (referenced by cli/bunfig.toml but missing from the public repo).

Validation

  • bun run typecheck (cli) clean.
  • 25 new tests + 152 related tests pass.

Note

DeepSeek V4 Flash (the default model) is multimodal: false and reads images via the server-side describe pipeline — for best results with photos, use MiniMax M3 / GPT-5.6 Luna / MiMo 2.5.

@codebuff-team

Copy link
Copy Markdown
Contributor

Good catches on terminal-images.ts: the missing m=0 terminator on the final chunk, repeating full control data (a=T, f=, c=, r=) on every chunk instead of just the first, and the iTerm2 size param using base64 length instead of decoded byte length — all three are genuine, well-described spec violations and worth porting.

But getKittyFormat() (terminal-images.ts) invents format ids that don't exist in the actual Kitty graphics protocol. Per the spec (https://sw.kovidgoyal.net/kitty/graphics-protocol/), f only takes 24 (RGB), 32 (RGBA), or 100 (PNG) — there is no 101/102/103/104 for PNG-with-alpha/JPEG/WebP/GIF. Since image-handler's compression path produces raw JPEG bytes (per your own new image-pipeline-integrity.test.ts), sending those bytes with f=102 to a real kitty/WezTerm/Ghostty terminal will not decode — the terminal has no such format and will either error or silently drop the image. This reintroduces exactly the 'distorted image' bug the PR is meant to fix, just for the compressed-JPEG code path instead of the chunking path. The tests you added only assert that the escape sequence contains f=102, not that any real terminal accepts it, so they don't catch this.

Before this can be ported: either convert compressed images to PNG before the kitty path (simplest, since 100 is the only guaranteed-compatible non-raw format), or drop the fabricated format ids and stick to 100/24/32 as the spec actually defines, deriving alpha/no-alpha via raw pixel mode if you want to avoid re-encoding.

The terminal-detection additions (WezTerm/Ghostty/Warp/Konsole) and the Windows pwsh fallback in clipboard-image.ts look solid independently and could be split into a smaller, separately reviewable PR.

@codebuff-team codebuff-team added bot:triaged Classified by the community triage bot pr:needs-work Right idea, not mergeable as written labels Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:triaged Classified by the community triage bot pr:needs-work Right idea, not mergeable as written

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants