feat(rocm): enable native GGUF kernels on ROCm (validated on RDNA4)

[zihaomu]

Draft follow-up to #132 and #133. This branch is based on the JIT follow-up head. Until its dependencies merge, GitHub Files changed also includes their diffs; the incremental GGUF review scope is the seven files listed below.

Summary

• stage GGUF CUDA sources in the extension cache before HIPify, keeping the checkout unchanged;
• key the staged source cache by source content and Torch/HIP versions;
• share FreeToken's ROCm target parser with the GGUF extension build;
• bridge FREETOKEN_ROCM_ARCH into PyTorch's internal PYTORCH_ROCM_ARCH build path so PyTorch does not inject its full default architecture list;
• accept only the declared wave32 RDNA3/RDNA4 targets and compile with -mno-wavefrontsize64, with an additional header-level fail-fast for toolchains exposing the wavefront macro;
• add numerical tests for Q4_0 dequantization, production ggml_mul_mat_vec_a8, and routed ggml_moe_a8_vec against PyTorch references.

Incremental review scope

Relative to zihaomu:feat/rocm-rdna4-jit:

• python/freetoken/kernel/gguf.py
• python/freetoken/kernel/utils.py
• python/freetoken/kernel/csrc/gguf/dispatch.h
• tests/kernels/test_gguf_rocm.py
• tests/utils/test_rocm_arch.py
• docs/install.md
• README.md

Diff size: 284 additions, 19 deletions.

Dependency

• Depends on feat(rocm): add RDNA3 and RDNA4 runtime foundation #132 for the ROCm/RDNA foundation.
• Depends on fix(rocm): make TVM-FFI index and store JIT kernels portable to HIP #133 for common HIP JIT launch/link support.
• Does not depend on the backend or RCCL follow-ups.

Validation

Hardware and software:

• AMD Radeon AI PRO R9700 (gfx1201)
• PyTorch 2.11.0+rocm7.14.0, HIP 7.14.60850

Fresh extension-cache result with only FREETOKEN_ROCM_ARCH=gfx1201 set:

• actual hipcc command contained only --offload-arch=gfx1201 plus -mno-wavefrontsize64;
• full source HIPify: 187 kernel launches replaced, 0 unsupported CUDA calls;
• hipcc compilation and shared-library link: PASS;
• build contract, Q4_0 dequantization, AMD int8-dot matvec, and two-expert MoE routing/numerics: 6 passed in 118.71 seconds;
• ROCm arch utility plus inherited JIT regression: 10 passed.

Support scope and known limitations

Native GGUF is source/build enabled for FreeToken's declared wave32 RDNA3/RDNA4 target set (gfx1100-gfx1103, gfx1200, gfx1201). The numerical validation in this PR was performed on gfx1201; RDNA3 GGUF hardware validation is not claimed yet. Non-RDNA/wave64 targets fail fast instead of risking silent reduction errors.

Q4_0 production paths are validated. Other GGUF quant formats and full model serving remain out of scope. This PR remains Draft until #132 and #133 establish the final base.

[andy-yang-1]

@zihaomu Thanks for your support! Can you DM me through email or through slack/discord/wechat? We can discuss more about FreeToken-AMD support.

[pheonixfirewingz]

I have an AMD XTX 7900 24GB to test it on RDNA 3 if you want ping me