Repository navigation
moonshine v2: per-layer attention bands and split encoder/decoder widths - #445
Merged
Merged
Conversation
MoonshineV2Config gains slidingWindows -- explicit (left, right) context per encoder layer in the HF encoder_config.sliding_windows convention. The German tiny streaming checkpoint uses [[17,5],[17,5],[17,1],[17,1],[17,5],[17,5]]: its middle layers look one frame ahead, which the edge-layer rule (lookahead on the first/last N layers, causal in between) cannot express. Null keeps the English tiny behaviour unchanged. decoderDim / decoderFfnDim let the decoder run at a different width than the encoder (the small checkpoints: 620 / 512); the adapter then carries the bias-free proj bridging the memory into decoder space, the encoder attention takes an explicit head dim when nHeads * headDim != dim, and the frontend takes the stream width as a parameter. Defaults are the English tiny values, so existing callers and the exported graphs are unchanged. Tests: MoonshineV2ConfigTest (common; JVM + linuxX64).
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two additions to the Moonshine v2 streaming model (
llm-inference/moonshine), both with defaults that keep the English tiny checkpoint and its exported graphs unchanged.Per-layer attention bands.
MoonshineV2Config.slidingWindowstakes explicit(left, right)context per encoder layer, in the HFencoder_config.sliding_windowsconvention. The German tiny streaming checkpoint uses[[17,5],[17,5],[17,1],[17,1],[17,5],[17,5]]: its middle layers look one frame ahead, which the existing edge-layer rule (lookahead on the first/last N layers, causal in between) cannot express.nullkeeps today's behaviour.slidingWindowForLayermaps the HF left context to the DSL window (left + 1).Split encoder/decoder widths.
decoderDim/decoderFfnDimlet the decoder run at a different width than the encoder (the small checkpoints: 620 / 512). The adapter then carries the bias-freeprojbridging the memory into decoder space, the encoder attention takes an explicit head dim whennHeads * headDim != dim, and the frontend takes the stream width as a parameter.Why
Without the per-layer bands the released module can describe the English checkpoint only; the German one needs them to export correctly.
Tests
MoonshineV2ConfigTestincommonTest(defaults = English tiny, explicit bands override the edge-layer rule, split widths);commonTestgetskotlin-test.:llm-inference:moonshine:jvmTest19/19,:llm-inference:moonshine:linuxX64Test3/3.Follow-up (not in this PR)
The StableHLO export and weight baking for Moonshine v2 live in
jvmTestonly, so they cannot be run from the published artifact. A main-source export CLI, likeFunctionGemmaExportCli, is the next step.