Intra-node compute compensation + overlap robustness (CUDA719 fix)

[ForFishes]

Summary

DeepEP-side changes for TeraMoE intra-node compute compensation plus run_overlap robustness under load skew.

Kernel / host

• CUDA 719 fix (high-skew GPU wedge): the fused-unzip chunk-completion counter now uses atom.acq_rel.gpu (atomic_add_acqrel_gpu in utils.cuh). The writer that publishes ready now inherits every peer writer's token / mapping-table release, so a consumer that acquires ready always observes complete data. One acquire per chunk — zero serialization, bitwise-identical on the normal path. Also adds fail-safe bounds guards on the fused-unzip writes.
• Dispatch peer-multicast plumbing (UnzipPeerBufs in configs.cuh; new default-nullptr params threaded through api.cuh / internode.cu / deep_ep.cpp / deep_ep.hpp) so migrated chunks can be direct-written into peer expert buffers. Default OFF — existing call paths are byte-for-byte unchanged.
• buffer.py: expose the new dispatch parameters.

Python (tests_overlap)

• test_internode_forward_backward.py: adaptive COMBINE_OVERLAP_RATIO = f(imbalance) (env ADAPTIVE_OVERLAP, default on) + a baseline-floor guard overlap_is_beneficial (env OVERLAP_BASELINE_FLOOR, default off) so overlap is never worse than baseline in losing regimes.
• compensation_planner.py: water-filling chunk-migration planner (expert-affinity batching, makespan guard, cost-aware no-op, 128-aligned fine-balance pass — reaches ~99% of the ideal upper bound).
• moe_compensated_node.py / moe_e2e_compensated.py: intra-node compute-compensation reference impl + end-to-end demo, bitwise-verified.
• test_dist_compensation.py: real distributed (per-rank GPU) compensation via NCCL NVLink migration; bitwise-identical fwd/dgrad.
• bench_three_configs.py: baseline vs overlap vs overlap+compensation benchmark.
• adaptive_controller_v3.py: unified adaptive scheduling / regime map.

Testing

• EP16 BF16 & FP8 correctness: bitwise diff=0 (FP8 wgrad cos ≈ 0.9993, expected requant).
• skew0.6 end-to-end: Exit 0 (previously reproduced CUDA 719).
• Compensation makespan speedup grows with imbalance and EP scale (bitwise-correct throughout).

Notes

• Peer-multicast ON path is not activated here — it is blocked on IPC symmetric-region allocation for the fused-unzip output buffers (needs an isolated cluster to validate deadlock-freedom). Off by default; zero regression.

🤖 Generated with Claude Code