Repository navigation
mlx-swift fork is 38 commits behind ml-explore/mlx-swift (mlx core 0.30.6 vs 0.32.2), and 4 upstream fixes were lost in a merge #204
Description
(Filing here because issues are disabled on SharpAI/mlx-swift.)
Summary
SwiftLM's mlx-swift submodule (b480, 1a2b3af) is current with the SharpAI fork, but the fork itself has fallen behind ml-explore/mlx-swift:
SharpAI/mlx-swift b480 |
ml-explore/mlx-swift 0.32.3 (2026-09-30) |
|
|---|---|---|
| Bundled mlx core | 0.30.6 | 0.32.2 |
| Divergence | merge base 49d73abc (#387). The fork is 62 commits ahead and 38 behind |
Catching up is a merge into the fork rather than a submodule bump in SwiftLM, since a bump can only move to a commit the fork already has.
What the gap costs on Apple Silicon
All of these are merged upstream:
| mlx PR | In | What it does |
|---|---|---|
| ml-explore/mlx#4241 | 0.32.1 | Dequantize in float32 (bf16 maths is emulated before M5). Upstream reports +20–24% prefill for 4-bit affine on M1 Max |
| ml-explore/mlx#4505 | 0.32.3 | Lower SDPA memory for head dim 256. Qwen3.5/3.6 full-attention layers are hd256 |
| ml-explore/mlx#4572, #4567 | 0.32.3 | gather_qmm / gather_mm improvements for MoE prefill |
| ml-explore/mlx#4020 | 0.32.3 | Native gated-delta-net Metal kernels, which could replace the hand-written kernel in mlx-swift-lm's GatedDelta.swift |
The mlx-swift fixes the fork is missing that matter most for a server are listed below. Each is in upstream history and none is in the fork:
- #461: deadlock between
CompiledFunction.lockandevalLock, and a data race on the compile cache - #472: pool Streams (Streams leak Metal resources), and fix Device inheritance
- #463:
StreamOrDevice.stream(_:)ignored its argument - #448:
FinalizerCaptureStateleak inMLXArray(rawPointer:…) - #398: unchecked
mlx_detail_compile/mlx_closure_applyreturn values - #471: wired-memory ticket lifecycle during cancellation
Four upstream fixes were lost in a merge
These commits are in the fork's history, but their code is missing from the current source. Each one's patch still applies cleanly to b480, and none of them reverse-applies. They appear to have been undone while resolving conflicts in a8f5092 ("chore: sync upstream and resolve SharpAI conflicts"):
- #358: wired-memory race condition (
WiredMemory.swift) - #363: dictionary key lookup in
Module.update(parameters:) - #367:
x[.ellipsis]returns self instead of a copy (#364) - #369: init-order dependency in
Module
To check one: git show 70346ae2 -- Source | git apply --check succeeds on b480.
Expected rebase friction
These are notes for whoever picks this up:
- Merge conflicts. Expect them where the fork's patches meet upstream changes: the TurboQuant additions at the end of
fast.h/.cpp(#4020 also appends there) andsdpa_vector.h, which several upstream SDPA PRs touch. - Generated code.
mlx-generatedand themoe_streamcodegen have to be regenerated. - Threading. Since mlx 0.31.2 every thread gets its own default stream, and Streams are now pooled (#472). Both could interact with the SSD-streaming thread pool, so that path needs a careful retest.
- Possible cleanup.
MLXFast.fromFp8looks redundant now that upstream wraps mlx'sfrom_fp8(#435).
Ask
Do you want an outside PR against SharpAI/mlx-swift for this, or would you rather do the sync yourselves, given that it touches the fork's own patches? I'm happy to do it as two PRs:
- Restore the four lost fixes. This is small and low risk.
- Merge
0.32.3, withrun_benchmark.shnumbers before and after.
The SwiftLM bump would then go through update_dependencies.yml as usual.
🤖 Generated with Claude Code
Activity
solderzzc commentedon Oct 5, 2026
Thanks for the detailed report. Most of this is already done; here is where each item stands against SharpAI/mlx-swift main (34bc52f).
ml-explore/mlx-swiftmain (b4f7684), merged as a real merge in Sync upstream ml-explore/mlx-swift (mlx v0.32.2, mlx-c ebc88f1) via real merge mlx-swift#20. The bundled mlx core is 0.32.2, which is what upstream bundles (includes mlx#4241). SwiftLM picked it up in chore: bump mlx-swift-lm to 4e6e521 and mlx-swift to 34bc52f (UserInput API) #211, together with the mlx-swift-lm sync (Sync upstream ml-explore/mlx-swift-lm (5e46681) via real merge mlx-swift-lm#74, still open).Transforms+Compile.swift, which has one fork-only addition (+13 lines, fix: run compiled functions on the task-scoped default device mlx-swift#17).MLXFast.fromFp8: fork-only and duplicates upstream'sfromFP8, but mlx-swift-lm still has 6 call sites, so it stays for now.If you want to take the core v0.32.3 bump (with before/after
run_benchmark.shnumbers), a PR against SharpAI/mlx-swift is welcome. AI-assisted analysis (Claude Code).CodeAndCanvas728 commentedon Oct 5, 2026
Thanks, that's a thorough check and it matches what I see here. I'll take the core v0.32.3 bump. The plan:
I expect conflicts in sdpa_vector.h, fast.h/.cpp, io/load.* and metal/device.cpp, and the PR will list how each was resolved. With 0.32.3 in, #4020 (mx.fast gated delta) can get its mlx-c and Swift bindings in a follow-up.
On fromFp8: if it helps, I can move the 6 mlx-swift-lm call sites to fromFP8 so the fork patch can go. Just say so.
solderzzc commentedon Oct 5, 2026
Sure, thanks, looking forward to it.
CodeAndCanvas728 commentedon Oct 6, 2026
Update on the core bump: SharpAI/mlx-swift#22 vendors mlx v0.32.3 (re-vendored with git, patches re-applied,
mlx-generatedregenerated, mlx-c left atebc88f1).--stream-experts): parity with 0.32.2 on TTFT and decode for the Baseline and TurboQuant KV configs. No speedup there; the 0.32.3 wins that target newer chips (#4505, #4567) don't show on this chip.floor_divide), which now builds seven ops where the MoE expert sort in mlx-swift-lm needs one for uint32 indices. The PR carries a small fix: skip the sign correction for unsigned integers.Package.swift, and the mlx-cgather_qmmcall passesnulloptfor the newglobal_scaleargument.The issue stays open for #4020 (
mx.fastgated delta), which needs its own mlx-c and Swift bindings. I'll file the #4515 finding upstream separately.AI-assisted analysis (Claude Code).
solderzzc commentedon Oct 6, 2026
Thank you for taking this on and for the thorough write-up. Bisecting the Metal timeout down to mlx #4515, keeping the fix small (unsigned only, with a test for both signs), and reporting honestly that this chip sees parity rather than a speedup is exactly the care a core bump needs. I am reading SharpAI/mlx-swift#22 now.
AI-assisted analysis (Claude Code).