Skip to content

fix(kllama): stop leaving packed-quant matmul scalar-only on iOS/macOS - #315

Merged
michalharakal merged 2 commits into
developfrom
fix/kllama-apple-native-kernels
Aug 31, 2026
Merged

michalharakal merged 2 commits into
developfrom
fix/kllama-apple-native-kernels

Conversation

@michalharakal

Copy link
Copy Markdown
Contributor

Summary

PlatformNativeKernels.ios.kt and .macos.kt were no-op stubs, both citing
SKaiNET#959 ("skainet-backend-native-cpu doesn't publish iosArm64/
iosSimulatorArm64/macosArm64 yet") as the reason. #959 shipped — the
engine has published real native-cinterop NEON kernels for those targets,
with runtime FEAT_DotProd dispatch, since then. linuxMain's actual already
calls installNativeKernels() to compensate for Kotlin/Native having no
ServiceLoader; the Apple actuals never got the same fix, so any app going
through kllama's registerPlatformBackends() facade has been silently
running packed-quant matmul on the scalar floor on iOS and macOS despite the
kernels being built and shipped.

Fix

Mirror linuxMain's actual on both files. kllama's nativeMain source set
already depends on skainet-backend-native-cpu and both iosMain/macosMain
dependsOn(nativeMain), so this is a code-only fix — no dependency wiring
needed.

Found while scoping out iOS support for the KernelRace sample in
SKaiNET-examples — this needed to land first, otherwise a new iOS target
would build and run correctly but silently stay scalar-only.

Test plan

  • compileKotlinJvm, compileKotlinLinuxX64, compileTestKotlinLinuxX64, compileAndroidMain, compileKotlinJs all pass
  • Apple targets (iosArm64/iosSimulatorArm64/macosArm64) — not verified, Kotlin/Native requires a macOS host to compile Apple targets at all, which this environment doesn't have. Verified by code review and cross-target compilation only.

PlatformNativeKernels.ios.kt and .macos.kt were no-op stubs, both citing
SKaiNET#959 ("skainet-backend-native-cpu doesn't publish iosArm64/
iosSimulatorArm64/macosArm64 yet") as the reason. #959 shipped — the
engine has published real native-cinterop NEON kernels for those targets,
with runtime FEAT_DotProd dispatch, since then. linuxMain's actual already
calls installNativeKernels() to compensate for Kotlin/Native having no
ServiceLoader; the Apple actuals never got the same fix, so any app going
through kllama's registerPlatformBackends() facade has been silently
running packed-quant matmul on the scalar floor on iOS and macOS despite
the kernels being built and shipped.

Mirror linuxMain's actual on both files. kllama's nativeMain source set
already depends on skainet-backend-native-cpu (build.gradle.kts:157) and
both iosMain/macosMain dependsOn(nativeMain), so this is a code-only fix,
no dependency wiring needed.

Verified: compileKotlinJvm, compileKotlinLinuxX64, compileTestKotlinLinuxX64,
compileAndroidMain, compileKotlinJs all pass. Apple targets (iosArm64/
iosSimulatorArm64/macosArm64) cannot be compiled on this Linux machine —
Kotlin/Native requires a macOS host for Apple targets — so this fix is
verified by code review and cross-target compilation only, not an actual
Apple-target build.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@michalharakal

Copy link
Copy Markdown
Contributor Author

Real-hardware validation of this fix (build + runtime kernel-tier check + before/after numbers) is tracked separately in #316, since it needs an actual Mac — not blocking this PR.

…iosMain

PR #315's installNativeKernels() call couldn't resolve on any Apple target
(compileKotlinIosArm64/IosSimulatorArm64/MacosArm64 all failed: Unresolved
reference 'installNativeKernels') — the commit message claimed nativeMain
already depended on skainet-backend-native-cpu, but the dependency was in
fact scoped to linuxMain's own dependencies block only, per the (now stale)
comment citing #298 as still-unresolved. #298 is done — the engine publishes
iosArm64/iosSimulatorArm64/macosArm64 variants of this artifact (confirmed:
~/.m2/repository/sk/ainet/core/skainet-backend-native-cpu-{iosarm64,
iossimulatorarm64,macosarm64}/0.51.0 all exist) — the build.gradle.kts wiring
just never moved off the pre-#298 linuxMain-only scoping.

Move the implementation(libs.skainet.backend.nativeCpu) declaration from
linuxMain's dependencies block up to nativeMain's, so linuxMain/macosMain/
iosMain all inherit it (they're already three siblings dependsOn(nativeMain)).

Verified on this Mac (closes the real-hardware half of #316):
- compileKotlinIosArm64/IosSimulatorArm64/MacosArm64 all succeed (previously
  failed on this exact branch).
- macosArm64Test and iosSimulatorArm64Test both green.
- Runtime check (KernelRegistry.availableNames(), throwaway test, run then
  deleted): reports [native-cinterop] on both macosArm64 and
  iosSimulatorArm64 after installPlatformNativeKernels() — not scalar.
- compileKotlinJvm/LinuxX64/Js + compileTestKotlinLinuxX64 still pass
  (the PR's original cross-target verification, re-confirmed unaffected).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@michalharakal

Copy link
Copy Markdown
Contributor Author

Validated on real Apple hardware (this needed a Mac, per the note above) and found + fixed a real bug: the original fix couldn't actually compile on any Apple target.

installNativeKernels() was unresolved on iosArm64/iosSimulatorArm64/macosArm64 — turns out implementation(libs.skainet.backend.nativeCpu) was still scoped to linuxMain's own dependencies block, not nativeMain, despite the commit message's claim. The engine side (#298/#959) is genuinely done — skainet-backend-native-cpu publishes iosArm64/iosSimulatorArm64/macosArm64 klibs now — but this module's build.gradle.kts never moved off the pre-#298 linux-only wiring. Pushed 9bb3707 moving the dependency up to nativeMain so all three Apple/Linux siblings inherit it.

After that fix, verified on this Mac:

  • compileKotlinIosArm64 / compileKotlinIosSimulatorArm64 / compileKotlinMacosArm64 all succeed
  • macosArm64Test and iosSimulatorArm64Test both green
  • Runtime check: KernelRegistry.availableNames() reports [native-cinterop] on both macosArm64 and iosSimulatorArm64 after installPlatformNativeKernels() — not scalar
  • compileKotlinJvm/LinuxX64/Js + compileTestKotlinLinuxX64 still pass (re-confirmed unaffected)

This covers the compile + runtime-registration checkboxes in #316. Real-device before/after throughput numbers still need actual hardware time, not just a compile+test pass — leaving that checkbox open there.

@michalharakal
michalharakal merged commit df04a0f into develop Aug 31, 2026
1 of 2 checks passed
@michalharakal
michalharakal deleted the fix/kllama-apple-native-kernels branch August 31, 2026 17:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant