Repository navigation
fix(kllama): stop leaving packed-quant matmul scalar-only on iOS/macOS - #315
Conversation
PlatformNativeKernels.ios.kt and .macos.kt were no-op stubs, both citing
SKaiNET#959 ("skainet-backend-native-cpu doesn't publish iosArm64/
iosSimulatorArm64/macosArm64 yet") as the reason. #959 shipped — the
engine has published real native-cinterop NEON kernels for those targets,
with runtime FEAT_DotProd dispatch, since then. linuxMain's actual already
calls installNativeKernels() to compensate for Kotlin/Native having no
ServiceLoader; the Apple actuals never got the same fix, so any app going
through kllama's registerPlatformBackends() facade has been silently
running packed-quant matmul on the scalar floor on iOS and macOS despite
the kernels being built and shipped.
Mirror linuxMain's actual on both files. kllama's nativeMain source set
already depends on skainet-backend-native-cpu (build.gradle.kts:157) and
both iosMain/macosMain dependsOn(nativeMain), so this is a code-only fix,
no dependency wiring needed.
Verified: compileKotlinJvm, compileKotlinLinuxX64, compileTestKotlinLinuxX64,
compileAndroidMain, compileKotlinJs all pass. Apple targets (iosArm64/
iosSimulatorArm64/macosArm64) cannot be compiled on this Linux machine —
Kotlin/Native requires a macOS host for Apple targets — so this fix is
verified by code review and cross-target compilation only, not an actual
Apple-target build.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Real-hardware validation of this fix (build + runtime kernel-tier check + before/after numbers) is tracked separately in #316, since it needs an actual Mac — not blocking this PR. |
…iosMain PR #315's installNativeKernels() call couldn't resolve on any Apple target (compileKotlinIosArm64/IosSimulatorArm64/MacosArm64 all failed: Unresolved reference 'installNativeKernels') — the commit message claimed nativeMain already depended on skainet-backend-native-cpu, but the dependency was in fact scoped to linuxMain's own dependencies block only, per the (now stale) comment citing #298 as still-unresolved. #298 is done — the engine publishes iosArm64/iosSimulatorArm64/macosArm64 variants of this artifact (confirmed: ~/.m2/repository/sk/ainet/core/skainet-backend-native-cpu-{iosarm64, iossimulatorarm64,macosarm64}/0.51.0 all exist) — the build.gradle.kts wiring just never moved off the pre-#298 linuxMain-only scoping. Move the implementation(libs.skainet.backend.nativeCpu) declaration from linuxMain's dependencies block up to nativeMain's, so linuxMain/macosMain/ iosMain all inherit it (they're already three siblings dependsOn(nativeMain)). Verified on this Mac (closes the real-hardware half of #316): - compileKotlinIosArm64/IosSimulatorArm64/MacosArm64 all succeed (previously failed on this exact branch). - macosArm64Test and iosSimulatorArm64Test both green. - Runtime check (KernelRegistry.availableNames(), throwaway test, run then deleted): reports [native-cinterop] on both macosArm64 and iosSimulatorArm64 after installPlatformNativeKernels() — not scalar. - compileKotlinJvm/LinuxX64/Js + compileTestKotlinLinuxX64 still pass (the PR's original cross-target verification, re-confirmed unaffected). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Validated on real Apple hardware (this needed a Mac, per the note above) and found + fixed a real bug: the original fix couldn't actually compile on any Apple target.
After that fix, verified on this Mac:
This covers the compile + runtime-registration checkboxes in #316. Real-device before/after throughput numbers still need actual hardware time, not just a compile+test pass — leaving that checkbox open there. |
Summary
PlatformNativeKernels.ios.ktand.macos.ktwere no-op stubs, both citingSKaiNET#959 ("
skainet-backend-native-cpudoesn't publishiosArm64/iosSimulatorArm64/macosArm64yet") as the reason. #959 shipped — theengine has published real
native-cinteropNEON kernels for those targets,with runtime FEAT_DotProd dispatch, since then.
linuxMain's actual alreadycalls
installNativeKernels()to compensate for Kotlin/Native having noServiceLoader; the Apple actuals never got the same fix, so any app goingthrough
kllama'sregisterPlatformBackends()facade has been silentlyrunning packed-quant matmul on the scalar floor on iOS and macOS despite the
kernels being built and shipped.
Fix
Mirror
linuxMain's actual on both files.kllama'snativeMainsource setalready depends on
skainet-backend-native-cpuand bothiosMain/macosMaindependsOn(nativeMain), so this is a code-only fix — no dependency wiringneeded.
Found while scoping out iOS support for the
KernelRacesample inSKaiNET-examples— this needed to land first, otherwise a new iOS targetwould build and run correctly but silently stay scalar-only.
Test plan
compileKotlinJvm,compileKotlinLinuxX64,compileTestKotlinLinuxX64,compileAndroidMain,compileKotlinJsall passiosArm64/iosSimulatorArm64/macosArm64) — not verified, Kotlin/Native requires a macOS host to compile Apple targets at all, which this environment doesn't have. Verified by code review and cross-target compilation only.