Skip to content

fix: detect nested MoE configuration - #117

Closed
Gwada wants to merge 1 commit into
SharpAI:mainfrom
Gwada:fix/detect-nested-moe-config
Closed

fix: detect nested MoE configuration#117
Gwada wants to merge 1 commit into
SharpAI:mainfrom
Gwada:fix/detect-nested-moe-config

Conversation

@Gwada

@Gwada Gwada commented Aug 5, 2026

Copy link
Copy Markdown

Summary

  • detect MoE expert counts stored under text_config
  • preserve the existing top-level num_local_experts behavior
  • add regression coverage for both configuration layouts

Problem

Some Hugging Face model configurations, including Qwen3.6 MoE checkpoints, store num_experts and num_experts_per_tok inside text_config. ModelProfiler only inspected top-level fields, so these models were classified as dense. As a result, their weight size was not measured correctly and --stream-experts was ignored.

Fix

Decode the nested expert fields in TextConfig and use them as fallbacks when the top-level values are absent. Top-level values keep priority, preserving existing behavior.

Validation

swift test --disable-sandbox --filter ModelProfilerTests

Executed 2 tests, with 0 failures.

The tests cover:

  • nested text_config.num_experts and text_config.num_experts_per_tok
  • the existing top-level num_local_experts format

Runtime validation with lmstudio-community/Qwen3.6-35B-A3B-MLX-8bit changed profiling from a false dense-model result to the expected MoE profile, allowing SSD expert streaming to activate.

@Gwada

Gwada commented Aug 5, 2026

Copy link
Copy Markdown
Author

Closing as a duplicate of #114, which covers the same nested text_config case and also supports the other expert-count key spellings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant