Skip to content

Adopt Foundation Models OS 27 surfaces: image attachments and capabilities - #241

Merged
mattt merged 3 commits into
mainfrom
os27-surfaces
Sep 14, 2026
Merged

mattt merged 3 commits into
mainfrom
os27-surfaces

Conversation

@mattt

@mattt mattt commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator

Carries @james-333i's commit from #206 on top of main, with authorship preserved.

On OS 27, Transcript image segments now map to Transcript.Segment.attachment with an ImageAttachment, built directly for file URLs and via ImageIO decoding for data sources. SystemLanguageModel exposes contextSize and supportsImageInput, which reports the OS 27 vision capability and returns false on earlier OS versions. The OS 27 references are behind #if compiler(>=6.4) with runtime availability checks, so the deployment floor is unchanged. James verified the image path end to end on an iPhone running iOS 27.

Two changes from #206:

Closes #206.

james-333i and others added 2 commits September 14, 2026 04:54
`SystemLanguageModel.contextSize` is back-deployed to OS 26.0
but first declared in the 26.4 SDK,
so Xcode 26.0 fails to compile the passthrough.
It is also unavailable on tvOS and watchOS.
Both are used solely by the OS 27 `ImageAttachment` extension,
so they belong inside its compiler gate.
@mattt
mattt merged commit 31eaa97 into main Sep 14, 2026
11 checks passed
@mattt
mattt deleted the os27-surfaces branch September 14, 2026 12:46
mattbpowell added a commit to mattbpowell/Snap-Shop that referenced this pull request Oct 14, 2026
Attachment(cgImage) SIGKILLed the app on betas 1-4. On the 27.0 release,
try alternative initializers (imageURL first, as verified on-device by
huggingface/AnyLanguageModel#241, then flattened sRGB CGImage, then raw
CGImage). The strategy in flight is persisted before construction; a
leftover marker on next launch marks it failed so a user crashes at most
once per strategy per OS build, then falls back to OCR automatically.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants