Repository navigation
feat(backend): SKEEP-007 Tier 1 -- skainet-backend-wasm-simd, the browser kernel pack - #1354
Merged
michalharakal merged 6 commits intoOct 16, 2026
Conversation
…ce kernel Adds the SKEEP-007 proposal (the browser lane: SIMD128 sidecar kernels, quantize-at-load) and lands its common-code tier: - matmulWeightTransposed on a canonical packed weight prefers an exact kernel for the row-major view, then the one-time cached relayout into kernel feed order served by the scalar packed kernels, and only then the decoding reference kernel. Every non-JVM target took the reference path on every call before. - a dense heap FP32 [out, in] weight is multiplied straight from its rows (4x4 tiles, one sequential accumulator per output, bit-identical to transpose-then-matmul) on tiers without a vectorised [k, n] kernel; preferDirectWeightTransposed() keeps the JVM and Apple tiers on their cached transpose. - NarrowFloatDenseTensorData implements RowDequantSource so 16-bit token tables are read row by row instead of widened at load. - ByteArrayRandomAccessSource in skainet-io-core (single array or chunks); JsBlobRandomAccessSource stops logging every read. - benchmarks/wasm-bench: a Kotlin/Wasm executable timing the matmul shapes of the shipped models; the Node baseline is recorded in the SKEEP page.
EncodingRequest.RequantizeTo has existed since SKaiNET-developers#1109 and the planner prices it, but only the GGUF loader executed it and only for BITNET_PLANES. Every Hugging Face checkpoint was dense FP32 on every target. - Q8_0Quantizer (ggml-faithful) beside Q4_0Quantizer, and the Requantizers registry (Q8_0, Q4_0, BITNET_PLANES) that loaders consult. - SafeTensorsParametersLoader / ShardedSafeTensorsParametersLoader take weightForm / weightFormFor; RequantizeTo packs every rank-2 float matrix whose rows block, leaves vectors and non-blocking rows dense, honours KERNEL_FEED, and refuses unknown encodings at construction. - The GGUF loader's planes-only requantize path serves every registered encoding, with feed-order relayout for Q8_0/Q4_0. - PlannerProfile.quantizeDenseTo + resolver rule (OUT_IN pinned) so a browser or phone profile can ask for it when a kernel can feed the result. - skainet-quantize in skainet-tensor-tools: SafeTensors -> GGUF with the matrices quantized offline, names kept, --keep prefixes written as F16.
…wser kernel pack Kotlin/Wasm has no SIMD types and no shared-memory threads, so the kernels live in a sidecar: the engine's own C kernel sources compiled by Emscripten into SIMD128 modules (single-threaded and pthreads), embedded in the klib as base64 constants and instantiated from a data:/blob: URL with the wasm bytes passed in -- Kotlin/Wasm does not propagate library resources to consumers. - WasmKernels: the bridge through Kotlin's own linear memory (no interop call per element); weights resident in the sidecar once per weight. - WasmRowMajorMatmulKernel + WasmSimdKernelPack keyed like the JVM FFM pack; WasmKernelProvider (priority 50); WasmCpuOps routes dense x.Wt through the new skainet_fp32_matmul_wt kernel; WasmSimd.install()/uninstall(). - WasmCpuOpsOverride in skainet-backend-cpu's wasmJs factory: the hook a pack has into context construction without ServiceLoader. - C: fp32_matmul_wt.c, __wasm_simd128__ bodies for the Q8_0/Q4_0 block dot products, SKAINET_HAVE_WASM_SIMD128 guard. - Node and ChromeHeadless parity tests; wasm-bench runs scalar, st and mt; kernel-support matrix gains the wasm-simd128 tier. Measured on Node: Laya Q8_0 matmuls 0.72 GFLOPS scalar -> 8.5 GFLOPS st -> 27 GFLOPS with four threads; one 22-layer decision ~1.2 s against 29 s.
Kotlin/JS keeps a Float local in double precision while the transpose-then- matmul path rounds through a Float32Array on every step, so the direct row-dot differs from it in the last bits there, as the golden tests already acknowledge for that target. The tiled-vs-definition tests keep pinning the bits where bits are stable.
…p-007-quantize-at-load
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SKEEP-007 Tier 1, stacked on #1353 (Tier 2) and #1352 (Tier 0): the browser kernel pack.
Why
Kotlin/Wasm exposes no SIMD types and no shared-memory threads, so no Kotlin kernel can be faster than a scalar loop on one core; the Tier 0 baseline measured 0.8–2.5 GFLOPS dense and 0.4–0.7 GFLOPS on packed Q8_0, about 29 s for one Laya decision. The engine already has the kernels — the C sources behind the JVM FFM and Android JNI packs. This PR compiles those same sources with Emscripten into a SIMD128 sidecar module and plugs it in behind the existing kernel-pack seam, so every browser build that calls
WasmSimd.install()gets them without application changes.What changes
skainet-backend-wasm-simd(new, wasmJs only).native/build-wasm.shcompilesskainet-backend-native-cpu/native/src(dense FP32/FP16/BF16, Q4_0/Q5_0/Q5_1/Q8_0/Q4_K/Q5_K/Q6_K, the row thread pool) with the pinned emsdk into two builds — single-threaded and pthreads — and embeds them as base64 constants (SidecarArtifacts.kt, generated and committed). Embedding rather than resources: Kotlin/Wasm copies a library's resources into neither a consumer's bundle nor its Node output, which is exactly what failed first. At runtime the glue is imported from adata:URL and the wasm bytes handed to Emscripten directly.WasmKernels: the bridge. Weights are uploaded into sidecar memory once per weight and stay resident; activations cross per call through Kotlin's own linear memory (kotlin.wasm.unsafe), staged by a wasm loop and copied with oneUint8Array.setper 4 MB chunk — no interop call per element.WasmRowMajorMatmulKernel+WasmSimdKernelPack: row-major kernels for the seven packed formats, keyed exactly likeFfmRowMajorMatmulKernel, soDefaultCpuOpsBase.matmulWeightTransposedfinds an exact kernel for the canonical view (Tier 0's first branch).WasmKernelProvider(priority 50): dense FP32 GEMM and the feed-order packed kernels, soKernelPacks.install()andchooseQuantizedMatmulHeapalso serve from SIMD128.WasmCpuOps:DefaultCpuOpsBasewith densex · Wᵀrouted through the newskainet_fp32_matmul_wtC kernel (row-dot over the weight's own rows; joins the native library and CMake for every tier). Everything else is the common implementation, and so is everything while the sidecar is not ready.WasmSimd.install()/uninstall(): loads the sidecar (pthreads whencrossOriginIsolated, or under Node), registers provider and pack, and setsWasmCpuOpsOverride— a new hook inskainet-backend-cpu's wasmJs factory, the one way a pack reaches context construction on a target withoutServiceLoader.__wasm_simd128__bodies for the Q8_0 and Q4_0 block dot products (the NEON structure on 128-bit lanes), aSKAINET_HAVE_WASM_SIMD128guard inskainet_simd.h; the other formats auto-vectorise from their portable bodies.KernelSupportMatrixTestgains thewasm-simd128tier;wasm-benchruns three passes (scalar, SIMD128 single-threaded, pthreads).Measurements (Node, same machine and shapes as the Tier 0 baseline; full table in the SKEEP page)
One Laya layer on Q8_0 is about 56 ms with pthreads: roughly 1.2 s for a 22-layer decision at 150 tokens, against 29 s on the scalar lane in the Tier 0 run.
Tests
WasmSimdParityTest(Node and ChromeHeadless): install swaps the ops and reports the SIMD mode; row-major Q8_0 and Q4_0 match the scalar lane within 1e-5 relative, the second call hits the resident weight, the pack is registered; densex · Wᵀ(rank 1 and 2) and the dense GEMM match the scalar lane.Known limits (follow-ups in the SKEEP page)
Storagekind) are Tier 1's second step, scoped by these measurements.install()pick the single-threaded build.Verification
wasmJsNodeTestandwasmJsBrowserTest(ChromeHeadless) of the new module: 3/3 each, pthreads build under Node.KernelSupportMatrixTestwith the new tier;generateKernelMatrixregeneratedkernel-support-matrix.adoc(JS/WASM column nowwasm-simd128).scripts/pr-gate.sh --quick(every module'sjvmTest+apiCheck): passed.native/build-wasm.shis reproducible from the pinned emsdk 4.0.14; the generatedSidecarArtifacts.ktcarries the wasm sha256 in its header.