Repository navigation
[no-ci] ci: plan builds and artifact reuse per bindings root - #3321
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
fdc9d18 to
ba05a1d
Compare
ba05a1d to
463bd2c
Compare
|
/ok to test 463bd2c |
|
463bd2c to
f1c0b26
Compare
f1c0b26 to
718fd32
Compare
718fd32 to
3fedd0d
Compare
|
/ok to test 88a2ab3 |
7418aad to
c1b95b6
Compare
|
— codex GPT-6 on behalf of Ralf —
Addressed with 4a0207a. This commit restores production’s nightly scope in #3321 and propagates it into #3322. The eight integration callers no longer run an additional standard-mode pass, and the dedicated standard caller is ARM64-only again. Additional coverage, including H200, can be considered separately from this migration.
These are configured job counts, not GPU-hours. The selected rows, modes and platforms match production; #3322 retains its source-aware artifact routing and security fixes. The comparison was checked locally, and all pre-commit checks passed. |
|
— codex GPT-6 on behalf of Ralf —
Clarified the PR description to distinguish package scope from matrix coverage.
The separate nightly workflow retains production’s integration and standard-test schedule following the correction for Major issue #1. |
|
/ok to test 29ca7e0 |
mdboom
left a comment
There was a problem hiding this comment.
It looks like this part of the simplification still hasn't been done. Let's do it (or explain why we can't):
Phase 2: planner and registry emit scalars where if: needs them.
Extend compute_ci_plan.py --github-output to also write flat outputs for platforms.linux, platforms.windows, windows_arm64_build, sdist_tests and core_api_checks.
In the same call, emit current-cuda-version and windows-arm64-*, so the three-mechanism registry step collapses into one.
Keep workplan and bindings-config as the JSON outputs for reusable workflows.
The checks job then reads needs.detect-changes.outputs.platform-linux and similar, with no fromJSON(...) || false chains.
Now that env-vars is just a simple wrapper around ci-env.py, let's remove it and update all users of env-vars to use the Python script.
|
— codex GPT-6 on behalf of Ralf —
A further simplification pass is implemented in 2676e38 and af6cd64 on #3321, and propagated into #3322 at a26b60e. Local validation passed. Full CI remains pending while we address the remaining review feedback. Changes and rationale
The resulting interface has eight outputs: two JSON payloads, five scheduler gates, and one existing single-major policy flag. Previously it had six. The simplification removes separate producers, serialized lists, wrappers, and repeated parsing; it does not reduce the total output count. The established baseline resolver and single-major build sequence were deliberately retained to keep this pass focused. Incremental diff statisticsNet changes on #3321, relative to its previous head:
Ordinary Git line counts, including comments and blank lines. One focused Windows ARM64 regression test was added; existing outcome assertions were adapted. Review and local validationAn independent Codex pass was followed by two review rounds with Fable 5.2, invoked with
These are local results. Full hosted CI has not yet been requested for these revisions. |
af6cd64 to
4d9f5ed
Compare
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
4d9f5ed to
e6dde0a
Compare
|
/ok to test e6dde0a |
— codex GPT-6 on behalf of Ralf —
Layer 6 of 7 of the staged "CUDA 12 bindings on
main" stack. Previous: #3319. Next: #3322. Stack map at the bottom.What this PR does
Teaches CI to do only the work a change needs, per bindings root. Before this layer, PR CI built and tested both majors in full.
ci/tools/compute_ci_plan.py: planning targets become(bindings, root),(core, cuNN),(python, root), andpathfinder. Acuda_bindings_12/edit rebuilds the CUDA 12 bindings and metapackage, reuses Core, Pathfinder, and CUDA 13 wheels from the baseline run at the exact merge base, and tests only CUDA 12 rows. Core edits rebuild both ABI halves. Shared or unknown paths build everything. A barev12.*orv13.*tag builds only its own line and uses published Pathfinder.ci/tools/select_test_matrix.py(new) replaces the jq program that was duplicated in both test-wheel workflows: per-row suite selection, local versus published floor bindings, runner-driver mapping, and a guard so a PR cannot silently lose a required local-wheel row. Job names now say what runs, for examplepathfinder+bindings12+core12+meta12 / py3.12, CTK12.9.1 (wheels; bindings PR).build-wheel.yml:ci/tools/ci_env.pyturns the workplan into per-root build and test variables; every step is gated per root; reusable artifacts are downloaded from the baseline and re-uploaded under the new SHA.ci/tools/ci_artifacts.pyshares artifact naming between producers, consumers, and baseline completeness checks. Inputscuda-version/prev-cuda-versionare replaced bybindings-config.ci.yml:ci-varsis folded intodetect-changes. One planner emits eight outputs:bindings-configandworkplanJSON, five job gates, and the Windows ARM64 single-major flag. The final status check uses the same job gates and rejects unexpected skips.ci/toolsfrom the workflow revision (.ci-control), so nightly can test an older artifact source with current tooling.ci/tools/select_pixi_packages.pyshares source-impact selection with the planner and also selects package-local Pixi manifests and documentation changes for the source-build smoke.After this merges
main, scheduled runs, and non-bindings tags still build all packages.ci.ymluses the PR wheel-test matrix, including for pushes tomain, matching production. The separateci-nightly.ymlworkflow retains production’s integration and standard-test schedule.mainthat becomes true after the first post-merge push run; before that, PRs build everything.Where to look
ci/tools/compute_ci_plan.py:source_impactsand the release-tag branch;test_compute_ci_plan.pystates the intended decisions as a table.ci/tools/select_test_matrix.py: short and self-contained.ci/tools/ci_env.pyand.github/workflows/build-wheel.yml: workplan-derived environments and artifact download/re-upload steps.All changes (27 files, +1,970 / -1,130)
Workflows:
build-wheel.yml,ci.yml,ci-nightly.yml,coverage.yml,ci-pixi-source-test.yml,test-sdist-linux.yml,test-sdist-windows.yml,test-wheel-linux.yml,test-wheel-windows.yml;.github/actions/fetch_ctk/action.yml.Tools:
compute_ci_plan.py(per-root impacts, execution records and scheduler outputs),select_test_matrix.py(new),ci_env.py(new: build/test environments; replacesenv-vars),ci_artifacts.py(new: shared artifact identity and baseline completeness),select_pixi_packages.py(new),bindings_config.py(Windows ARM64 configuration),cuda_core_bindings_floor.py,run-tests(published Pathfinder for focused releases).Tests:
test_compute_ci_plan.py,test_select_test_matrix.py(new),test_select_pixi_packages.py(new),test_merge_cuda_core_wheels.py(new),test_bindings_config.py,ci/tools/tests/AGENTS.md(new: proportional test guidance).Configuration and release guidance:
ci/versions.yml,.github/RELEASE-bindings.md.Validation
463bd2c3(run 41197068718).3fedd0d7dfb5179924c89d6c508ba6aae6ad2e65: 369 CI-tool tests and configured workflow lint passed. Full-stack CI on #3322 passed at126ed74fb5bbfbc91a04e50557cf936efef4da7f(105 successful jobs; one expected skip).Stack map: cumulative capabilities by layer
v12.*tagcuda_bindings_12/; the PR-time CPU source-build smoke covers both roots; three CUDA 12 runtime fixes are incuda.coreand the metapackage still consume12.9.xartifacts12.9.xbranchv12.9.Ntag onmain; CUDA 13 releases are unchanged12.9.xis frozenLayers #3314 and #3315 have merged. The remaining five migration PRs (#3317, #3318, #3319, #3321 and #3322) will land together using GitHub’s atomic stack merge, with full CI on #3322 as described in the landing plan. #3434 follows separately, after a successful post-migration
mainCI run supplies complete artifacts for both CUDA majors.