[ajrasane]

What does this PR do?

Type of change: Bug fix

The Qwen3-VL and Qwen3.5 FP8 vision recipes previously quantized vision-block Linears whose 3D attention inputs are flattened by vLLM's FlashInfer FP8 GEMM, causing vision encoder initialization to fail. Narrow the shared visual quantizer rules to primary and deepstack merger Linears, whose inputs are 2D. Update both recipe descriptions, the recipe guide, the 0.48.0 changelog entry, and the quantizer-selection and checkpoint-export tests. This changes existing recipe output: vision blocks now remain in high precision and affected checkpoints must be regenerated.

Usage

from modelopt.recipe import load_recipe

recipe = load_recipe("model_type/qwen3_5/ptq/fp8_vision-kv_none")
quant_cfg = recipe.quantize.model_dump()

Testing

• Verified the four Qwen3-VL/Qwen3.5 recipe-selection cases fail with the previous quantizer rules and pass with this change (4 passed).
• Ran the four GPU calibration and unified-HF-export cases for vision-only and joint FP8 recipes (4 passed).
• Exported a Qwen3.5 vision-language checkpoint after four image-calibration samples on a GB300. The export marks vision blocks as excluded and retains FP8 merger scales. Unmodified vLLM 0.31.0 started successfully, returned HTTP 200 from /health, and answered an image prompt correctly.
• Ran all applicable pre-commit hooks on the changed files, including recipe validation.

Before your PR is "Ready for review"

Make sure you read and follow Contributor guidelines and your commits are signed (git commit -s -S).

Make sure you read and follow the Security Best Practices (e.g. avoiding hardcoded trust_remote_code=True, torch.load(..., weights_only=False), pickle, etc.).

• Is this change backward compatible?: ❌ — rerunning either existing recipe now leaves vision blocks in high precision; previously exported checkpoints are unchanged.
• If you copied code from any other sources or added a new PIP dependency, did you follow guidance in CONTRIBUTING.md: N/A
• Did you write any new necessary tests?: ✅
• Did you update Changelog?: ✅

Additional Information

This avoids the failing 3D FP8 path by reducing vision quantization coverage. It does not change vLLM's FP8 kernel or repair already exported checkpoints. End-to-end serving used four calibration images; a full-size calibration run with the changed recipe was not repeated.

Summary by CodeRabbit

• Bug Fixes
  • Updated Qwen3-VL and Qwen3.5 FP8 vision recipes to quantize primary and deepstack vision-merger layers while keeping vision blocks and patch embeddings in high precision for vLLM compatibility.
  • Recipes that also enable language-model FP8 quantization retain that option while keeping vision blocks and patch embeddings in high precision.
  • Calibration checks now account for vision-merger input quantizers.
  • Re-quantize affected checkpoints before deployment to apply the updated settings.
• Documentation
  • Clarified which model components are quantized by the FP8 vision recipes.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Repository: NVIDIA/Model-Optimizer/.coderabbit.yaml
• Review profile: CHILL
• Plan: Enterprise
• Run ID: e75c9fd1-be1a-48c9-aaa6-002208eaef93

📥 Commits

Reviewing files that changed from the base of the PR and between 60716cf and 2efb835.

📒 Files selected for processing: 1

• CHANGELOG.rst

🚧 Files skipped from review as they are similar to previous changes: 1

• CHANGELOG.rst

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

📝 Walkthrough

Qwen3-VL and Qwen3.5 FP8 vision quantization now targets vision-merger Linears. Vision blocks remain high precision. Recipe descriptions, calibration validation, tests, and the changelog reflect the updated scope.

Changes: Qwen vision FP8 recipes

Layer / File(s)	Summary
Restrict visual FP8 quantizers modelopt_recipes/model_type/qwen3_vl/ptq/vision_fp8.quant_cfg.yaml, examples/hf_ptq/hf_ptq.py, tests/unit/recipe/test_qwen_vision_recipe.py, tests/gpu/torch/export/test_qwen_vision_recipe_export.py	FP8 rules target visual merger Linears. Calibration validation matches visual merger input quantizers. Unit and export tests check the enabled quantizers and verify that vision-block scales are absent.
Align recipe descriptions modelopt_recipes/model_type/qwen3_vl/ptq/fp8_vision*.yaml, modelopt_recipes/model_type/qwen3_5/ptq/fp8_vision*.yaml, modelopt_recipes/ptq.md, CHANGELOG.rst	Recipe descriptions and documentation specify vision-merger FP8 quantization and high-precision vision blocks. The changelog advises re-quantizing affected checkpoints before deployment.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 2efb8

No actionable issue is established for this change; it is mergeable after normal checks.

Pre-merge checks: 5 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 3 files. (1 skipped: 1…	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 5 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main change: keeping Qwen vision blocks in high precision for vLLM.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Security Anti-Patterns		PASS. The only changed example Python logic updates recipe quantizer-name matching; it adds no torch.load(..., weights_only=False), numpy.load(..., allow_pickle=True), hardcoded `trust_remote_code…

Full details: Security Anti-Patterns

Explanation

PASS. The only changed example Python logic updates recipe quantizer-name matching; it adds no torch.load(..., weights_only=False), numpy.load(..., allow_pickle=True), hardcoded trust_remote_code=True, external-input eval/exec, or # nosec. No modelopt package Python files or dependency manifests changed, and no new dependency was added.

Full details: Docstring Coverage

Explanation

Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 3 files. (1 skipped: 1 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[cjluo-nv]

Bot review (gpt-6.1-sol) — DM the bot to share feedback.

LGTM: the narrowed rules preserve merger FP8 coverage and disable vision blocks; the justified existing-test updates require human sign-off.

Needs action:

• Confirm the behavior-aligned assertion changes in tests/unit/recipe/test_qwen_vision_recipe.py and tests/gpu/torch/export/test_qwen_vision_recipe_export.py, then give final approval.

No action needed:

• Design review: this avoids incompatible 3D vision-attention inputs using existing shared recipe rules, rather than Python quantizer overrides or new OmegaConf/Pydantic configuration plumbing. The PR explains why it narrows coverage instead of repairing vLLM kernels.
• Test edits are justified: exact merger-only selection replaces broader enablement assertions, and export now checks that block scales are absent. Language-model and KV-cache checks remain intact.
• Recipe ordering keeps blocks and patch embeddings disabled in both vision-only and joint recipes. Documentation explicitly requires checkpoint regeneration.
• Tests were inspected, not executed during this review.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.78%. Comparing base (200011e) to head (2efb835).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2822      +/-   ##
==========================================
+ Coverage   69.68%   78.78%   +9.10%     
==========================================
  Files         657      657              
  Lines       73283    73283              
==========================================
+ Hits        51065    57735    +6670     
+ Misses      22218    15548    -6670     

Flag	Coverage Δ
examples-diffusers	20.11% <ø> (-0.05%)	⬇️
examples-gpt-oss	13.51% <ø> (-0.04%)	⬇️
examples-hf_ptq	23.26% <ø> (-0.06%)	⬇️
examples-llm_distill	13.59% <ø> (-0.04%)	⬇️
examples-llm_eval	17.27% <ø> (-0.04%)	⬇️
examples-llm_qat	17.52% <ø> (-0.03%)	⬇️
examples-llm_sparsity	15.79% <ø> (-0.06%)	⬇️
examples-megatron_bridge	26.52% <ø> (-0.25%)	⬇️
examples-specdec_bench	13.29% <ø> (-0.04%)	⬇️
examples-speculative_decoding	17.51% <ø> (-0.09%)	⬇️
examples-torch_onnx	21.48% <ø> (-0.08%)	⬇️
examples-torch_trt	15.25% <ø> (-0.04%)	⬇️
examples-vllm_serve	15.00% <ø> (-0.04%)	⬇️
gpu	59.34% <ø> (+37.41%)	⬆️
unit	60.19% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-10-16 18:48 UTC