perf(moe): default-on MLXCEL_FUSED_MOE, validated on M5 (#282)

[inureyes]

Summary

Flips MLXCEL_FUSED_MOE to on by default for single-token decode-MoE, after validating the fused kernels on M5 Max (Neural Accelerator) hardware for the first time. The #268 series (#276/#278/#279/#280/#281) shipped these kernels off-by-default and only ever ran them on M1 Ultra; this enables them by default once confirmed clean on M5. Closes #282.

The fused path stays an env override: MLXCEL_FUSED_MOE=0 (also false/off/no, case-insensitive) forces the proven gather_qmm / SwitchGLU path. This also fixes a latent bug. The previous gate was std::env::var("MLXCEL_FUSED_MOE").is_ok(), which is true whenever the variable is set, so MLXCEL_FUSED_MOE=0 enabled the kernel instead of disabling it. A new switch_layers::fused_moe_enabled() helper centralizes the decision (default-on; an explicit falsey value disables) and all four call sites route through it.

M5 validation (the core of this issue)

Apple M5 Max (Mac17,7, 128 GB, macOS 26.5.1; is_m5_neural_accelerator() returns true), release build with metal+accelerate, greedy temp-0, measured against the true gather_qmm baseline (env unset):

Model	activation	bits	decode tok/s (off to on)	gain	greedy parity	kernel dispatched
gemma-4-26b-a4b-it	GeGLU	4	133.9 to 142.2	+6.2%	within f16 jitter, coherent	yes (30/tok)
qwen3.5-35b-a3b	SwiGLU	4	153.6 to 161.3	+5.1%	within f16 jitter, coherent	yes (40/tok)
qwen3-30b-a3b	SwiGLU	4	60.5 to 61.7	+1.9%	byte-identical	yes (48/tok)
dots.llm1	SwiGLU	4+6	14.1 to 14.0	~par	byte-identical	yes (61/tok)

• No NaN and no garbage on any model. The documented M5-Max mixed-dtype NaN risk (in the separate C++ fused_moe_forward) does not materialize in the new moe_gateup / moe_down / GeGLU kernels, which cast scores to the activation dtype and accumulate in f32.
• No regression anywhere. Gains are smaller than M1 Ultra because the Neural Accelerator already accelerates the baseline expert matmul, leaving less inter-kernel idle for the fusion to recover.
• Kernel execution on M5 was confirmed per layer with a temporary dispatch trace (reverted before this commit), not merely inferred from output, because byte-identical parity alone cannot distinguish a real run from a silent fallback.
• M5 is clean with real gains, so the default is flipped for all hardware; no is_m5_neural_accelerator() gate is needed.

Notable findings surfaced during validation

• MLXCEL_FUSED_MOE=0 never disabled the kernel (the .is_ok() bug above). Fixed here, which is exactly the off switch the issue asked to preserve.
• qwen1.5-moe could not be validated: it loads as qwen2_moe, whose SparseMoeBlock was never wired with the fused dispatch, so it always falls back to gather_qmm (0 dispatches). Only four paths actually dispatch the kernel: qwen3_moe, qwen3_next, dots1, gemma4. mixtral, olmoe, minimax, phimoe, lfm2, and qwen3_vl_moe reuse SwitchGLU but are not wired. The doc's "Models covered" list overstated coverage and is corrected here. Wiring the remaining families is a separate follow-up.

Changes

• src/models/switch_layers.rs: add fused_moe_enabled() (default-on; 0/false/off/no disables) plus a fused_moe_enabled_from unit test.
• src/models/{qwen3_moe,qwen3_next,dots1,gemma4}.rs: route the decode-MoE gate through fused_moe_enabled() instead of std::env::var("MLXCEL_FUSED_MOE").is_ok().
• docs/benchmark_results/fused-moe-decode-kernel-design.md: flags table now default-on, roadmap step 5 marked done, new M5 results section, corrected "Models covered".

Testing

• M5 real-checkpoint validation on the four wired families (table above), all coherent with no NaN.
• Behavioral flip check on gemma4: default (unset) = 142.5 tok/s and produces the fused output; MLXCEL_FUSED_MOE=0 = 133.4 tok/s and produces the gather_qmm baseline output; the two outputs differ as expected, confirming the flip and the fixed =0 semantics.
• cargo test --release switch_layers, cargo fmt, cargo clippy --release --features metal,accelerate -- -D warnings.

Closes #282.