feat: add simple ROCm GPU support for RDNA3 (gfx1100-1103) - #23
Conversation
- Add hip_compat.h shim mapping CUDA runtime API to HIP equivalents - Update pinned_tensor.cpp to compile under both nvcc and hipcc - Add ROCm detection in arch.py (is_rocm, get_rocm_gfx_arch, is_gfx11xx_family) - Guard NVIDIA arch checks to return None on ROCm - Skip nvcc version check in _toolchain.py when on ROCm - Add ROCm build path in setup.py (ROCM_HOME, amdhip64, --offload-arch) - Add _hip_cflags() in kernel/utils.py for JIT compilation on ROCm - Add is_rocm() and driver_hip_version() in backend.py - Add rocm-smi fallback in __main__.py for clangd generation - Add TODO(ROCm) for NCCL->RCCL, flashinfer/sgl_kernel ROCm builds, Triton autotune RDNA3 tuning, PDL equivalent, hiprtc JIT cache - Add AMD ROCm classifier in pyproject.toml
|
Could add ROCm support for RDNA4 GPUs as well? |
|
For RDNA4 users, the complete validated adaptation is available on This is the monolithic, runnable RDNA4 snapshot used for validation on an AMD Radeon AI PRO R9700 ( git clone --branch feat/rdna4-rocm --single-branch https://github.com/zihaomu/FreeToken.gitSee the branch's ROCm source-install instructions, including the tested ROCm 7.14 image and For upstream review, the same work is being organized as the standalone foundation #132 and Draft follow-ups #133, #134, #135, and #136. Those PRs are review-sized splits; the branch linked above remains the current single-branch version for RDNA4 users. |
|
Following the base-chain discussion, I have opened a standalone current- |
Four knobs measured neutral before the round produced anything, and the cause was the A/B harness rather than the knobs: the arms never let the hot adapter tick, so any knob that changes what the adapter aims at could not show an effect. Production sits within about 5 points of the hindsight-optimal hot set while the arms sat 28 to 47 points off it, which is the tell. With ticks forced on both arms, splitting the prefill and decode histories moved post-document decode from 12.5 to 17.4 tok/s and collapsed major faults per step from 472 to 126, at the cost of 23 percent slower prefill. The capacity policy and empty-skip stay neutral, the first on its own profiled ceiling of half a point. Also records the essay harness noise floor of roughly 10 to 20 percent, measured by an ABBA ordering, which is why rounds two and three could not have resolved the 1 to 5 percent effects they were chasing. Refs: FlashML-org#19, FlashML-org#20, FlashML-org#21, FlashML-org#23 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A88MCbnLtwsFSHmqwuJezY
3 participants
ROCm support for RDNA3 GPUs
Adds initial ROCm support so FreeToken builds and runs on AMD RDNA3 cards (gfx1100-1103).
Supported GPUs:
What works:
pinned_tensor.cppcompiles under hipcc via a hip_compat.h shim that maps CUDA runtime API to HIPLeft as TODO(ROCm) in the code: