[ajrasane]

What does this PR do?

Type of change: Bug fix

ONNX AutoTune's TensorRT Python benchmark fails during initialization on TensorRT 11 because it accesses the removed NetworkDefinitionCreationFlag.EXPLICIT_BATCH enum. Create the network with STRONGLY_TYPED alone: explicit batching is already implicit in TensorRT 10 and 11, and strong typing remains enabled on both versions.

Add a parametrized GPU regression test that constructs the real benchmark, verifies its network creation flags, and builds and measures tiny INT8 and FP8 Q/DQ MatMul candidates. Update the AutoTune guide's TensorRT Python example to use the same supported network flags, and document the integration-test fixture. The same constructor exists in ModelOpt 0.47, so this also includes a changelog entry. BuilderFlag.DIRECT_IO remains available in the tested versions and needs no change.

Usage

from modelopt.onnx.quantization.autotune import TensorRTPyBenchmark

benchmark = TensorRTPyBenchmark(warmup_runs=1, timing_runs=2)
latency_ms = benchmark.run("model.onnx")

Testing

• Before the fix, both new regression cases fail at the real constructor with the missing EXPLICIT_BATCH AttributeError.
• TensorRT 11.1.0.106, RTX 6000 Ada: pytest tests/gpu/onnx/quantization/autotune/test_benchmark.py -q — 36 passed, including INT8/FP8 candidate builds and measurements, Python benchmark bytes/path/callable cases, and trtexec smoke tests.
• TensorRT 10.13.3.9, RTX 6000 Ada: pytest tests/gpu/onnx/quantization/autotune/test_benchmark.py -k TestTensorRTPyBenchmark -q — 5 passed.
• Re-ran the original minimal constructor reproduction on TensorRT 11: initialization succeeds.
• Repository pre-commit checks on the changed files pass, including Ruff, mypy, and Bandit.
• Verified the corrected guide's network creation on TensorRT 11.1.0.106 and 10.13.3.9; both initialize successfully with strong typing enabled.

Full standalone AutoTune searches and quantization CLI accuracy evaluations were not run. Candidate measurements here validate the shared benchmark path, not model accuracy.

Before your PR is "Ready for review"

Make sure you read and follow Contributor guidelines and your commits are signed (git commit -s -S).

Make sure you read and follow the Security Best Practices (e.g. avoiding hardcoded trust_remote_code=True, torch.load(..., weights_only=False), pickle, etc.).

• Is this change backward compatible?: ✅
• If you copied code from any other sources or added a new PIP dependency, did you follow guidance in CONTRIBUTING.md: N/A
• Did you write any new necessary tests?: ✅
• Did you update Changelog?: ✅
• Did you get Claude approval on this PR?: N/A

Additional Information

TensorRT 10-to-11 migration guide

Summary by CodeRabbit

• New Features
  • Added imatrix calibration for GGML formats, including weighted calibration using per-column and per-expert statistics. Existing final codes remain unchanged, allowing calibration of weights skipped by GPTQ.
• Bug Fixes
  • Fixed ONNX AutoTune benchmark initialization with TensorRT 11 while retaining strongly typed network support for TensorRT 10.
  • Fixed LayerNorm input quantization in presets and recipes that enable input quantizers.
• Documentation
  • Updated the TensorRT Python API example to use strongly typed network configuration.
• Tests
  • Added GPU integration coverage for INT8 and FP8 quantized MatMul benchmarks, including finite, positive latency checks. FP8 coverage is skipped on unsupported GPUs.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Repository: NVIDIA/Model-Optimizer/.coderabbit.yaml
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 66124d70-7bf8-4bf4-a036-6264ea2c5213

📥 Commits

Reviewing files that changed from the base of the PR and between 0fc4792 and d2b55c3.

📒 Files selected for processing: 1

• CHANGELOG.rst

🚧 Files skipped from review as they are similar to previous changes: 1

• CHANGELOG.rst

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

📝 Walkthrough

The ONNX AutoTune benchmark and guide now create TensorRT networks with only the STRONGLY_TYPED flag. An integration test checks INT8 and FP8 quantized MatMul models. The changelog records the TensorRT 11 benchmark initialization fix.

Changes: AutoTune Network Initialization

Layer / File(s)	Summary
Strongly typed network creation and quantized-model validation modelopt/onnx/quantization/autotune/benchmark.py, tests/gpu/onnx/quantization/autotune/test_benchmark.py, docs/source/guides/9_autotune.rst, CHANGELOG.rst	The benchmark and guide remove EXPLICIT_BATCH and retain STRONGLY_TYPED. The integration test validates INT8 and FP8 quantized MatMul models, checks that the network is strongly typed, and asserts finite positive latency. The FP8 case skips GPUs below compute capability 8.9. The changelog records the TensorRT 11 initialization fix.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Suggested reviewers: gcunhase

Merge Risk: ⚪ Minimal · up to d2b55

The TensorRT 11 initialization fix has no identified merge-blocking issue; merge after normal checks.

Pre-merge checks: 6

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main change: fixing ONNX AutoTune benchmark initialization for TensorRT 11.
Docstring Coverage		Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 2 files. (1 skipped: 1 …
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Security Anti-Patterns		No listed security anti-pattern was introduced. The only changed modelopt Python line sets the TensorRT STRONGLY_TYPED flag. No examples Python files or dependency files changed. Added lines con…

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-10-16 13:16 UTC

[cjluo-nv]

Bot review (bedrock-claude-opus-5-5) — DM the bot to share feedback.

The code fix is correct and has a good GPU regression test, but the Autotune guide still has the same EXPLICIT_BATCH snippet that breaks on TensorRT 11.

Needs action:

• Update the "Using TensorRT Python API" example in docs/source/guides/9_autotune.rst to use only 1 << int(trt.NetworkDefinitionCreationFlag.STRONGLY_TYPED). As written, the snippet raises the same AttributeError this PR fixes.

No action needed:

• benchmark.py was the only code path still using EXPLICIT_BATCH. The change is right for TensorRT 10 and 11, since explicit batch is implicit on both and STRONGLY_TYPED is kept.
• The new parametrized test runs the real constructor, checks the STRONGLY_TYPED flag, and builds and measures INT8 and FP8 Q/DQ MatMul engines. It skips FP8 below SM89. No existing tests were changed.
• The changelog entry is under 0.49.0 Bug Fixes, which is the right place.

[ajrasane]

Addressed the review in 9a5a621: the “Using TensorRT Python API” example now creates the network with STRONGLY_TYPED alone, matching the benchmark implementation. Verified the corrected network creation on TensorRT 11.1.0.106 and 10.13.3.9; both succeed with strong typing enabled. Pre-commit checks pass.

Also added a concise docstring to the integration-test class's TensorRT/CUDA fixture in response to the docstring coverage warning. The PR description now includes the documentation change and its validation. Full AutoTune searches and CLI accuracy evaluation remain explicitly untested.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.67%. Comparing base (f3678b6) to head (d2b55c3).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2795      +/-   ##
==========================================
+ Coverage   69.33%   78.67%   +9.33%     
==========================================
  Files         654      655       +1     
  Lines       72665    72776     +111     
==========================================
+ Hits        50385    57256    +6871     
+ Misses      22280    15520    -6760     

Flag	Coverage Δ
examples-diffusers	20.17% <0.00%> (-0.06%)	⬇️
examples-gpt-oss	13.53% <0.00%> (+0.01%)	⬆️
examples-hf_ptq	23.32% <0.00%> (-0.02%)	⬇️
examples-llm_distill	13.61% <0.00%> (+0.01%)	⬆️
examples-llm_eval	17.31% <0.00%> (+0.03%)	⬆️
examples-llm_qat	17.56% <0.00%> (-0.04%)	⬇️
examples-llm_sparsity	15.83% <0.00%> (-0.01%)	⬇️
examples-megatron_bridge	26.64% <0.00%> (-0.15%)	⬇️
examples-specdec_bench	13.31% <0.00%> (+0.02%)	⬆️
examples-speculative_decoding	17.55% <0.00%> (-0.08%)	⬇️
examples-torch_onnx	21.54% <0.00%> (-0.08%)	⬇️
examples-torch_trt	15.28% <0.00%> (-0.01%)	⬇️
examples-vllm_serve	15.02% <0.00%> (+0.10%)	⬆️
gpu	59.54% <100.00%> (+37.71%)	⬆️
unit	59.93% <0.00%> (+0.05%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[gcunhase]

This matches TrtExecBenchmark with --stronglyTyped enabled.

LGTM.