ci: combine SwiftBuddy DMG builder into release workflow and clean root directory - #46
Merged
Merged
Conversation
solderzzc
added a commit
that referenced
this pull request
Aug 9, 2026
Points at SharpAI/mlx-swift-lm#46, which makes the Gemma 4 assistant's callAsFunction delegate to the trunk. Without it this PR's feature aborts on any prompt over prefillStepSize (512 tokens) with Fatal error: Layer 0 is a KV-shared layer but received no sharedKV because MTPTokenIterator.prepare() prefills through context.model, which this PR makes the assistant — and an assistant checkpoint is entirely KV-shared layers that cannot run without sharedKV from the trunk. To be re-pointed at main once #46 lands, since the squash rewrites the SHA. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
solderzzc
added a commit
that referenced
this pull request
Aug 10, 2026
…ly (#137) * feat: make --mtp work for model families that ship MTP heads separately --mtp has been silently a no-op for Gemma 4. The gate is `context.model is any MTPLanguageModel`, and only Qwen35Model, Qwen35TextModel and DeepseekV4Model conform — those carry their MTP heads inside the main checkpoint. Gemma 4 does not: Google ships the heads as a separate assistant checkpoint, and Gemma4AssistantModel conforms to DualModelMTP (MTPLanguageModel plus a back-reference to the trunk it drafts for). Nothing in Sources/ ever set that reference except Gemma4MTPBench, which is not a target in Package.swift and so cannot build — leaving the whole path unreachable. --mtp-assistant-model loads the assistant, injects mainModelRef, and routes through the existing generateMTP call. Rather than adding a second generation branch, mtpContext() picks which context generateMTP should run against: the main context for in-checkpoint MTP, or a derived context whose model is the assistant while tokenizer, processor and configuration — and the KV cache passed alongside — stay the trunk's. That mirrors the reference usage in Gemma4MTPBench and keeps one code path, so the prompt cache is unaffected. An explicit flag rather than an id table: the table in #109 maps gemma-4-e4b-it to the E2B assistant and gemma-4-31b-it to the 26B one, which look like slips, and a wrong guess here silently drafts from the wrong model. Measured, and the result is not favourable yet. Output is correct — identical prefixes to baseline — but throughput is worse on both pairs available here: gemma-4-e2b-it-4bit + E2B assistant: 136.8 → 117.2 tok/s gemma-4-26b-a4b-4bit + 26B assistant: 74.1 → 63.6 tok/s and flat across --num-mtp-tokens 1/2/3 (63.3 / 64.1 / 63.6 on the 26B pair). Invariance to draft depth points at a fixed per-round cost rather than draft token cost, which is what the unlanded maxSharedKV=16 cap in #109 targets. Both assistants also ship bf16 against 4-bit trunks, so each drafted token costs more than the trunk token it replaces. So this makes the flag mean something and gives the perf work something to be measured against; it is not a speedup on its own. MTP stays opt-in and off by default, and with no --mtp-assistant-model the behaviour is byte-identical to before. 259 tests pass. Refs #109. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: bump mlx-swift-lm to pick up the dual-model MTP prefill fix Points at SharpAI/mlx-swift-lm#46, which makes the Gemma 4 assistant's callAsFunction delegate to the trunk. Without it this PR's feature aborts on any prompt over prefillStepSize (512 tokens) with Fatal error: Layer 0 is a KV-shared layer but received no sharedKV because MTPTokenIterator.prepare() prefills through context.model, which this PR makes the assistant — and an assistant checkpoint is entirely KV-shared layers that cannot run without sharedKV from the trunk. To be re-pointed at main once #46 lands, since the squash rewrites the SHA. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test: exercise chunked prefill with a prompt over the 512-token boundary Every prompt in this repo's test suite is under 80 characters, so prepare() always returned prompt tokens without forwarding them and chunked prefill was never run. That gap is how a dual-model MTP crash on any real-sized prompt reached a green CI (SharpAI/mlx-swift-lm#46) — the failure needed only a prompt past prefillStepSize to appear, and nothing in CI supplied one. Adds one ~2700-token request to the contract suite. An empty response is treated as a failure, not an error case: a crash in prefill drops the connection rather than returning an error body, which is precisely the signature being watched for. This covers the ordinary generate path only. CI runs no --mtp job, so the speculative variant of the same code path remains uncovered (#128). Verified locally: server logs prompt=2697t for the new case, suite 10 passed 0 failed 2 skipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore: re-point mlx-swift-lm at the merged prefill fix on main SharpAI/mlx-swift-lm#46 landed as squash commit 6a2c179, which replaces the branch SHA the previous bump pointed at. The tree is byte-identical to the interim pointer, so the CI already run against this PR still applies — only the commit identity changes, from a now-deleted branch to main. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
As requested, this integrates the DMG packaging phase natively into the release CI, preventing release fragmentation. It also updates the autogenerated Release Notes to explicitly detail DMG usage instructions. Finally, this cleans up the repository root by moving dozens of scrap files to docs/profiling and tests/sandbox.