Repository navigation
Bump transformers dependency to >=4.57,<5.15 - #2050
Merged
Merged
Conversation
Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
Contributor
📝 WalkthroughWalkthroughThe update raises the minimum supported Transformers version to 4.57, extends the upper bound to below 5.15, updates test pins, and removes Qwen3-VL version guards and lazy imports. ChangesTransformers compatibility
Estimated code review effort: 2 (Simple) | ~10 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 6✅ Passed checks (6 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
modelopt/torch/puzzletron/anymodel/models/__init__.py (1)
27-27: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winRemove the dead version imports.
This change removes the only use of
_Versionand_transformers_versionin this module. Delete those imports so the module does not retain dead code.As per coding guidelines, remove dead code, including unused imports.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@modelopt/torch/puzzletron/anymodel/models/__init__.py` at line 27, Remove the now-unused _Version and _transformers_version imports from the module containing the qwen3_vl wildcard import, while preserving the qwen3_vl export.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@modelopt/torch/puzzletron/anymodel/models/__init__.py`:
- Line 27: Remove the now-unused _Version and _transformers_version imports from
the module containing the qwen3_vl wildcard import, while preserving the
qwen3_vl export.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: d6f4e7f8-1841-4431-b400-d9bebbb2fe89
📒 Files selected for processing (7)
CHANGELOG.rstmodelopt/torch/__init__.pymodelopt/torch/puzzletron/anymodel/models/__init__.pynoxfile.pypyproject.tomltests/_test_utils/torch/transformers_models.pytests/gpu/torch/puzzletron/test_puzzletron.py
💤 Files with no reviewable changes (1)
- tests/gpu/torch/puzzletron/test_puzzletron.py
Contributor
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2050 +/- ##
===========================================
+ Coverage 66.94% 77.92% +10.97%
===========================================
Files 519 519
Lines 59401 59400 -1
===========================================
+ Hits 39767 46287 +6520
+ Misses 19634 13113 -6521
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
Collaborator
|
what is the standard/process of bumping the version? |
Collaborator
Author
We bump transformers to latest every release. Since we're already at 5.15, I also removed 4.56 as some models require optional imports till 4.57 to clean up. We might drop 4.x at some point too |
Collaborator
|
OK, so 4.57 seems the last version of v4? Looking forward to just drop v4 sometime in the future. |
shengliangxu
approved these changes
Aug 3, 2026
kevalmorabia97
added
the
cherry-pick-done
kevalmorabia97
added a commit
that referenced
this pull request
Aug 4, 2026
#2041 #2031 #2061 #1983 #1628 (#2062) ## Cherry-picked PRs - #2022 - #2026 - #2032 - #1981 - #2010 - #2043 - #2042 - #2038 - #2050 - #2041 - #2031 - #2061 - #1983 - #1628 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added recipe-driven ONNX quantization and export for embedding and reranking models. * Added launcher support for inline commands, package requirements, and configurable Docker users. * Added QAD workflows, Nemotron data recipes, pruning, quantization, and evaluation examples. * Added evaluation accuracy and pruning score thresholds. * **Bug Fixes** * Improved ONNX control-flow conversion, compressed NVFP4 export, dataset loading, and multi-GPU DiffusionGemma handling. * **Documentation** * Updated CUDA 12/13 installation guidance and vLLM Blackwell deployment instructions. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: vipandya <vipandya@nvidia.com> Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com> Signed-off-by: Fridah-nv <201670829+Fridah-nv@users.noreply.github.com> Signed-off-by: weimingc <17592131+meenchen@users.noreply.github.com> Signed-off-by: ajrasane <131806219+ajrasane@users.noreply.github.com> Signed-off-by: Meng Xin <mxin@nvidia.com> Signed-off-by: Chenjie Luo <chenjiel@nvidia.com> Signed-off-by: Chenjie Luo <108829653+cjluo-nv@users.noreply.github.com> Signed-off-by: Juhi Mittal <juhim@nvidia.com> Signed-off-by: Gwena Cunha <4861122+gcunhase@users.noreply.github.com> Signed-off-by: Gwenaelle Cunha Sergio <gcunhasergio@nvidia.com> Co-authored-by: vishalpandya1990 <vishalpandya1990@gmail.com> Co-authored-by: Frida Hou <201670829+Fridah-nv@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Wei-Ming Chen <17592131+meenchen@users.noreply.github.com> Co-authored-by: Ajinkya Rasane <131806219+ajrasane@users.noreply.github.com> Co-authored-by: OpenAI Codex <noreply@openai.com> Co-authored-by: mxinO <164952785+mxinO@users.noreply.github.com> Co-authored-by: Chenjie Luo <108829653+cjluo-nv@users.noreply.github.com> Co-authored-by: Juhi Mittal <39641197+juhi10071998@users.noreply.github.com> Co-authored-by: Gwena Cunha <4861122+gcunhase@users.noreply.github.com>
kevalmorabia97
added a commit
that referenced
this pull request
Oct 6, 2026
### What does this PR do? Type of change: new feature (dependency bump), bug fix, backward breaking (DBRX removal) Extends the supported transformers range to `>=4.57,<5.19` and moves CI's `tf_latest` to 5.18 (the latest release). It also fixes everything in 5.15–5.18 that breaks ModelOpt; each fix keeps 4.57–5.14 working. Separately, it removes DBRX support, and the customized-model quantization guide now uses Llama 4 as its example. Among other things, the HF side can now load `glm5_next` (GLM-5.3-Flash, native since 5.16.1). - **Pins:** the `pyproject.toml` `hf` extra, the "not tested" warning in `modelopt/torch/__init__.py`, and `noxfile.py` `tf_latest` (`~=5.14.0` → `~=5.18.0`). - **HF tensor-parallel quantization (5.16 rewrote TP on DTensors, #47579):** - 5.16 removed the module-level `_hf_tp_plan` that ModelOpt used to recognize TP-sharded linears. A model loaded with `tp_plan="auto"` was therefore quantized as if unsharded: its forward failed on mixed `Tensor`/`DTensor` inputs. - Sharded linears are now recognized by a DTensor weight on the model's TP mesh (`model._device_mesh`): `Shard(0)` is colwise and `Shard(1)` is rowwise. Matching the mesh (by `==`) keeps FSDP2 shards out. The mesh is found on any submodule, so a model held by a trainer, PEFT or a wrapper still works, and ModelOpt warns if a TP mesh exists but no linear matched. - HF now installs its TP input/output transforms as an instance `forward`, which `DynamicModule.convert` keeps as `_forward_pre_dm`. `HFParallelLinear.forward` routes through it, and sets HF's `_hf_quantized_needs_local_tp` so the quantized linear runs on this rank's plain weight shard, in training as well. - The weight-access dispatch in `core_utils.py` now keys on the layer's own `enable_weight_access_and_writeback` instead of `_hf_tp_plan`. - **DBRX support removed (internally agreed):** the DBRX quantization plugin, its model spec, its unified-export preparation handler, the TRT-LLM export branches, and the AutoQuantize rules for its rewritten experts. This also drops the 5.0–5.14 vs 5.15+ expert-layout split that 5.15 would otherwise have needed. The customized-model quantization guide now uses the shipped `Llama4TextExperts` plugin (fused `torch.bmm` experts) as its example instead of DBRX. There is a Deprecations entry in the changelog. - **`TrainingArguments.warmup_ratio` removed (5.15, #46917):** - `ModelOptArgParser` now accepts either spelling in YAML configs and translates to whichever the installed transformers supports. `warmup_ratio` becomes `warmup_steps` on 5.15+, where a value below 1 is a ratio. A fractional `warmup_steps` becomes `warmup_ratio` on 4.x, where `warmup_steps` is an int. - The `llm_qat` train configs now use `warmup_steps`. They failed to parse on 5.15+. - `examples/alpamayo/qad.py` picks the supported key. - **Puzzletron Qwen3-VL (5.17, #48105):** `Qwen3VLMoeVisionRotaryEmbedding` now takes the vision config instead of `dim`, so the descriptor builds it from the config there. The old call raised `AttributeError: 'int' object has no attribute 'rope_parameters'`. - **Alpamayo:** the prompt cache is cropped with a negative count, since positive absolute-size `crop()` calls are deprecated (#47720). - **Liger eval in ModelOpt's trainers:** 5.15+ adds `skip_logits=True` to eval inputs when `use_liger_kernel` is set. - ModelOpt computes the fused (KD) loss itself from hidden states, and its `compute_loss_func` makes the Trainer pop `labels`. So the student's and the teacher's Liger forwards raised `skip_logits is True, but labels and shift_labels are None`. - This failed `llm_qat` QAT, QAD and QLoRA in CI. The flag is now dropped in both paths. **Checked, no change needed:** - **T5 (#47014):** T5 now dispatches through `ALL_ATTENTION_FUNCTIONS`, so it registers with the generic `_QuantAttention`. FP8 KV-cache scales calibrate on 5.18, and `_T5QuantAttention` stays for 4.57–5.14. - **Indexer layer types (#48974):** remapped to `indexed_attention`. ModelOpt never compares these strings. - **`min_pixels` / `max_pixels` (#49021):** ModelOpt passes them only to `from_pretrained`, which is still supported. - **No ModelOpt usage:** the removed mask functions, `update_candidate_strategy`, and `use_mamba_kernels`. ### Usage ```bash pip install -U "nvidia-modelopt[hf]" # now resolves transformers 5.18 ``` ### Testing All on this host (Python 3.13, torch 2.14). The 3.12 nox session can't start here because the system Python has a libffi/`_ctypes` mismatch, which is unrelated. - **`nox -s "unit-3.13(torch_214, tf_latest)"` on 5.18, before the fixes:** 4532 passed, 1 failed (`test_dbrx`, whose model is now removed). - With the fixes, `tests/unit/torch/quantization` + `tests/unit/torch/opt` on 5.18: 1414 passed, 8 skipped. - `tests/unit/torch/quantization/plugins/test_huggingface.py`: passes on 5.14.1 and 5.18.0. - **`tests/gpu/torch/quantization/plugins/test_transformers_tp.py` (2 GPUs):** passes on 5.14.1, 5.16.1 and 5.18.0. - It now also asserts that every sharded decoder linear is quantized as a TP-aware layer. - Without the fix it fails on 5.18 with `aten.mm.default got mixed torch.Tensor and DTensor`. - **`tests/examples/llm_qat/test_llm_qat.py` on 5.18 (2 GPUs):** QAT on DDP, QAD on FSDP2, and QLoRA pass. - QAT and QAD ran with `attn_implementation: sdpa` locally, because `flash-attn` doesn't build on this host. - Without the `skip_logits` fix, QAD on FSDP2 reproduces the CI error on both ranks. - **`test_transformers_tp` wrapped-model case:** fails with the old top-level mesh lookup and passes with the fix, on 5.14.1 and 5.18.0. - **After removing DBRX:** `tests/unit/torch` on 5.18: 3355 passed, 15 skipped. The `test_dbrx` cases are gone, the spec tests that used DBRX now use `qwen3_5_moe`, and the DBRX cases in the export-registry test are dropped. - **Guide snippet:** on 5.18, with ModelOpt's own Llama4 plugin unregistered, the guide's `_QuantLlama4TextExperts` inserts the same 4 quantizers on a tiny `Llama4TextExperts` and matches the shipped plugin's FP8 output exactly. - **Warmup keys:** - New `test_yaml_warmup_spelling_follows_transformers` covers both translation directions. - Parsing `llm_qat/configs/train/qat_nvfp4.yaml` and a legacy `warmup_ratio: 0.05` YAML through `llm_qat`'s `TrainingArguments` on 5.18 gives `warmup_steps 0.05` for both. - `ARGUMENTS.md` regenerates unchanged. Its pre-commit hook couldn't run on this host because the hook's `uv` env has no torch, so I generated it manually and diffed. - **Puzzletron:** on a tiny Qwen3-VL-MoE, `init_rotary_embedding` reproduces the model's own vision `inv_freq` on both 5.14.1 and 5.18.0. - **Alpamayo crop:** keeps exactly the prefill tokens on both 5.14.1 and 5.18.0. - **Not run locally:** the full GPU and example suites, including the Puzzletron GPU test. CI covers them with the new `tf_latest` pin. ### Before your PR is "*Ready for review*" - Is this change backward compatible?: ❌ DBRX is no longer supported. Otherwise, behavior on transformers 4.57–5.14 is unchanged and 5.15–5.18 are newly supported. `llm_qat` configs move from `warmup_ratio` to `warmup_steps`, but `ModelOptArgParser` still accepts `warmup_ratio` on every version. - If you copied code from any other sources or added a new PIP dependency, did you follow guidance in `CONTRIBUTING.md`: N/A - Did you write any new necessary tests?: ✅ Warmup-key translation, and the TP test now asserts TP-aware conversion (also for a wrapped model). - Did you update [Changelog](https://github.com/NVIDIA/Model-Optimizer/blob/main/CHANGELOG.rst)?: ✅ A Deprecations entry for the DBRX removal. The upper-bound bump itself gets none, as in #2050. - Did you get Claude approval on this PR?: ❌ ### Additional Information `uv.lock` is left to the weekly relock job. Example-specific pins (`examples/speculative_decoding`, `examples/puzzletron`, `examples/windows/*`) are left as they are; they pin for their own reasons. Follow-up opportunities from the new releases, not in this PR: - Real tiny `glm5_next` / `step3p7` models in the recipe tests, which would need a skip on the 4.57 CI leg. - HF-side KDA quantization (KimiLinear is native since 5.17). - NemotronH Omni support in `hf_ptq`. - Measuring `hf_ptq` calibration speed now that linear-attention kernels are opt-in (#47630). 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Compatibility** * Expanded supported Hugging Face Transformers versions to 4.57–5.18. * Added compatibility with newer Transformers tensor-parallel layers and Qwen3-VL vision embedding APIs. * Improved compatibility with Liger fused-loss evaluation during training and distillation. * **Bug Fixes** * Improved handling of warmup settings across Transformers versions. * Corrected sequence cropping during Alpamayo quantization. * **Breaking Changes** * Removed built-in DBRX quantization and TensorRT-LLM export support. DBRX quantization remains available in ModelOpt 0.47; custom plugin support can be added. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Transformers Dependency bump - Drop 4.56 (still support 4.57 with deprecation note) and extend to 5.14 (nemo:26.08 ships with this version)
tests/{gpu_megatron,examples/megatron_bridge}in nemo:26.08.rc3Summary by CodeRabbit
Compatibility
Documentation
Testing