Add GGML MoEExperts (Ming-Image) + fix F32 tokenizer_json tensors

[bbirdkt]

Problem

Loading the Ming-Image GGUF text encoders (Ling-Mini-2.0 17.3B MoE) fails with:

GGMLOps has no attribute 'MoEExperts'

Recent comfy core added comfy.ops.MoEExperts, and comfy.text_encoders.ming_image.BailingExperts routes its expert banks (experts.gate_up_proj.weight, experts.down_proj.weight) through moe_experts_forward, which requires the bank_resident / expert_linear / expert_weight interface. GGMLOps had no implementation of it, so every Ming GGUF TE fails at load time.

A second, independent failure follows after that: the tokenizer_json tensor inside these GGUFs is stored as F32 with one float value per byte (the byte stream of the HF tokenizer.json encoded as floats), while the core Ming tokenizer expects raw utf-8 bytes — it currently errors with UnicodeDecodeError instead.

Fix 1: GGMLMoEExperts (ops.py)

A GGMLLayer over the full packed expert bank (logical shape [num_experts, out_features, in_features], exposed as-is by the loader):

• bank_resident(input) dequantizes the bank once through the standard cast_bias_weight path (so it goes through the usual dequant + patch machinery), then expert_linear(input, i) indexes into the resident bank. This is the hot path used by moe_experts_forward.
• expert_weight(i, input=...) lazily dequantizes a single expert: experts are contiguous in the packed buffer, so it slices the expert's byte range and dequantizes it as a 2D [out_features, in_features] matrix. This avoids materializing the full bank when used without the context manager.
• Constructor signature matches the core MoEExperts (num_experts, in_features, out_features, bias, device, dtype) and shape mismatches fail loudly. Also exposed as GGMLOps.MoEExperts.

Verified on a real Q4_K bank (thinker.layers.1.mlp.experts.* from Ming-Image-0.1-Ling-Mini-2.0-Q4_K_M.gguf, E=256):

• per-expert slice dequant is bit-exact vs full-bank dequantization (torch.equal) for both gate_up_proj and down_proj;
• bank_resident + expert_linear matches a manual F.linear reference (max diff 0.0);
• the real comfy.text_encoders.llama.moe_experts_forward runs end-to-end over both banks and returns the expected [tokens, hidden] output.

Fix 2: F32 tokenizer_json tensors (loader.py)

gguf_clip_loader now converts a tokenizer_json tensor stored as F32-per-byte into a plain uint8 byte tensor, so the core tokenizer can .decode("utf-8") it. Validated against the actual file: all 12,210,709 values are integral, and the converted bytes parse as valid tokenizer JSON (vocab size 156,891).

Testing

• ComfyUI v0.37.0 (commit 88ab4a06), RTX 3090 24GB, python 3.10 / torch 2.14+cu126.
• Ming-Image-0.1-Design Q4_K_M diffusion + Ming-Image-0.1-Ling-Mini-2.0 Q4_K_M TE through UnetLoaderGGUF + CLIPLoaderGGUF(type=ming): full t2i workflow completes in 27s, output is a valid 1024×1024 RGBA PNG.
• Regression: Qwen-Image 2.1 (int8) and Ideogram 4 (int8, dual model) workflows still complete normally after the patch (90s / 95s).

Notes

• No existing GGUF-loading path is changed except for F32 tokenizer_json tensors, which previously could not be consumed at all.
• Bias support is included (dequant + per-expert bias), though the current Ming config creates these layers with bias=False.

[therordion]

Independent verification from a third setup — both issues this PR targets are confirmed against the published files; the fix direction looks right.

Setup: ComfyUI pinned at 3b4c0b0e ("feat: ming-image support #16482"), node city96/ComfyUI-GGUF at db80272 (PR #484), encoder from realrebelai/Ming-Image_GGUFs: Text Encoders (UPDATED)/Ming-Image-0.1-Ling-Mini-2.0-Q2_K.gguf, 7,947,560,832 bytes, SHA256 21e5e950186583cb770a4b48063f455fd971ced935d78439065c12592db2f4b6 (matches the repo LFS oid bit-for-bit).

Tokenizer tensor (read with gguf 0.19.0): tokenizer_json, ggml type 0 (F32), shape [12210709], every value integral (min 10, max 197) — one character code per float instead of one byte. Stock _MingRawTokenizer (comfy/text_encoders/ming_image.py, lines 263-269) does tensor.numpy().tobytes().decode("utf-8"), which yields 48,842,836 bytes of little-endian floats and raises UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf6 in position 2: invalid start byte (first value 123.0f = 00 00 f6 42). Narrowing float32 to uint8 recovers a valid tokenizer.json (11,126,241 chars; tokenizers.Tokenizer.from_str accepts it).

MoEExperts: comfy/ops.py declares MoEExperts only inside the mixed_precision_ops factory (lines 1348-1349; nested class at 1507). It is absent from comfy.ops, comfy.ops.manual_cast and comfy.ops.disable_weight_init (verified dynamically on the pinned tree), while CLIPLoaderGGUF.load_patcher passes model_options["custom_operations"] = GGMLOps — so loading a Ming-Image text encoder raises AttributeError: type object 'GGMLOps' has no attribute 'MoEExperts' from ming_image.py:106-107.

Both failures were reproduced independently; with a local prototype of this PR's two fixes (a GGMLLayer-based MoEExperts plus F32-to-uint8 conversion in gguf_clip_loader) a full Ming t2i run completes at 1024x1024 in 33.06 s (12 steps, 2.51 s/it) on an RTX 5070 Ti Laptop (12 GB) with --lowvram.

One data point to double-check: after decoding, my tokenizer reports vocab size 157,179 (identical with and without added_tokens=True), while this PR's description mentions 156,891 — possibly a different revision of the encoder file; worth confirming which artifact the PR was validated against.