feat(gpu): derive sandbox requirements from CDI specs by elezar · Pull Request #2265 · NVIDIA/OpenShell

elezar · 2026-07-14T13:11:48Z

Summary

Derive sandbox filesystem and process requirements from resolved CDI GPU specs so Docker GPU sandboxes can carry the effective device, mount, and group policy into the supervisor.

Related Issue

Related: #1606

Changes

Add shared CDI context and resolver helpers for policy-relevant device nodes, mounts, and additional GIDs.
Upload Docker CDI context/spec mounts when GPU CDI devices are selected.
Enrich sandbox policies with CDI-derived paths and supplemental process groups.
Apply CDI-derived additional GIDs before dropping supervisor privileges.
Add architecture and user-facing documentation for CDI-derived GPU sandbox policy behavior.

Testing

mise run pre-commit passes
Unit tests added/updated
E2E tests added/updated (if applicable)

Checklist

Follows Conventional Commits
Commits are signed off (DCO)

copy-pr-bot · 2026-07-14T13:11:53Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

github-actions · 2026-07-14T13:19:23Z

🌿 Preview your docs: https://nvidia-preview-pr-2265.docs.buildwithfern.com/openshell

github-actions · 2026-07-20T13:58:00Z

Label test:e2e-gpu applied for 5cd240a. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute GPU E2E after building the required supervisor image once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

github-actions · 2026-07-20T13:58:02Z

Label test:e2e applied for 5cd240a. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

Signed-off-by: Evan Lezar <elezar@nvidia.com>

drew · 2026-07-22T14:36:52Z

        openshell_core::sandbox_env::TELEMETRY_ENABLED.to_string(),
        openshell_core::telemetry::enabled_env_value().to_string(),
    );
+    if include_cdi_context {


[P1] Prevent container images from supplying their own CDI context. This only inserts OPENSHELL_CDI_CONTEXT when GPUs are selected, so a non-GPU container does not override an OPENSHELL_CDI_CONTEXT value inherited from the selected image. A crafted image can embed its own context and CDI spec; the supervisor accepts any nonempty context path, converts CDI deviceNodes into read-write Landlock paths without verifying that they are device nodes, and applies additionalGids. That lets an image expand its effective policy even though no trusted CDI context was uploaded. Please explicitly override this variable with an empty value for non-CDI containers. As defense in depth, also restrict the supervisor to the fixed context path/spec mount root and verify device-node types before granting read-write access.

elezar

gator-agent

PR Review Status

Validation: This PR is project-valid because it implements roadmap issue #1606 for Docker GPU/CDI sandbox policy derivation, includes user-facing documentation, and has the required E2E/GPU E2E labels.
Head SHA: c040b8dd5d7adcb126d1c5199e6203ba5dfdc93b

Thanks @drew, I checked the CDI context trust-boundary concern you raised and confirmed it is still a blocking issue on this head. The independent review also found related supervisor-side hardening gaps and a Docker token cleanup regression.

Review findings:

Blocking findings remain on CDI context trust, CDI device-node validation, and failed-provisioning token cleanup.

Docs: Fern docs were updated under docs/; no navigation change appears necessary for these existing pages.
Checks: Required checks are currently green, including DCO, Branch Checks, Helm Lint, Core E2E, and GPU E2E.
E2E: test:e2e and test:e2e-gpu are applied and passing.

Next state: gator:in-review

elezar · 2026-07-22T15:34:15Z

        openshell_core::sandbox_env::TELEMETRY_ENABLED.to_string(),
        openshell_core::telemetry::enabled_env_value().to_string(),
    );
+    if include_cdi_context {


gator-agent

Critical: OPENSHELL_CDI_CONTEXT is only inserted for GPU/CDI requests. Docker inherits image ENV, so a non-GPU image can provide its own CDI context path and cause the supervisor to apply image-controlled CDI paths and supplemental groups. Please set this variable for every container, using the fixed context path only when CDI is expected and an empty value otherwise.

elezar · 2026-07-22T15:34:15Z

+fn resolve_cdi_requirements_from_env(
+    writable_file_allowlist: &HashSet<String>,
+) -> Result<Option<openshell_core::cdi::CdiDerivedRequirements>> {
+    let Ok(context_path) = std::env::var(openshell_core::sandbox_env::CDI_CONTEXT) else {


gator-agent

Critical: The supervisor currently accepts any non-empty OPENSHELL_CDI_CONTEXT path. Even with Docker clearing the env for non-CDI containers, this should fail closed unless the path is the driver-owned fixed context path, and context spec dirs should be constrained to the supervisor CDI spec root.

elezar · 2026-07-22T15:34:15Z

+    F: Fn(&str) -> Option<CdiPathKind>,
+{
+    for device_node in &edits.device_nodes {
+        let path = normalize_policy_path(&device_node.path)?;


gator-agent

Critical: CDI deviceNodes are normalized but not verified as device nodes before they become read-write Landlock paths. A malformed or malicious spec could list a regular file and expand write policy. Please verify char/block device type before adding a CDI device node path.

elezar · 2026-07-22T15:34:15Z

-            }
            DockerProvisioningFailure::new("ContainerCreateFailed", status.message())
        })?;
+        write_sandbox_token_file(sandbox, &self.config)


gator-agent

Warning: Token file creation now happens before container creation, but the later create/upload/start failure paths no longer clean it up. Please restore cleanup_sandbox_token_file on each post-write provisioning failure path so failed sandboxes do not leave credential artifacts behind.

Signed-off-by: Evan Lezar <elezar@nvidia.com>

elezar

gator-agent

PR Review Status

Validation: This PR is project-valid because it implements roadmap issue #1606 for Docker GPU/CDI sandbox policy derivation, includes architecture and Fern documentation updates, and has the required test:e2e and test:e2e-gpu labels.
Head SHA: cf6a99519f85611da0558468ce42f84c6940976d

Thanks @drew, I re-checked the CDI context trust-boundary concern you raised against the latest head. The Docker env cleanup and token cleanup paths are improved, but the independent review found the supervisor still needs a stronger driver-attestation boundary before CDI-derived policy is trusted.

Review findings:

Blocking finding remains on CDI context trust/attestation.
Warning remains on explicit CDI device ID namespace validation.

Docs: Fern docs were updated under docs/; no navigation change appears necessary for these existing pages.
Checks: Current head has DCO and Helm Lint passing, with Branch Checks, E2E, and GPU E2E still pending/queued.
E2E: test:e2e and test:e2e-gpu are applied; results are still pending for this head.

Next state: gator:in-review

elezar · 2026-07-22T16:32:23Z

+fn resolve_cdi_requirements_from_env(
+    writable_file_allowlist: &HashSet<String>,
+) -> Result<Option<openshell_core::cdi::CdiDerivedRequirements>> {
+    let Ok(context_path) = std::env::var(openshell_core::sandbox_env::CDI_CONTEXT) else {


gator-agent

Critical: resolve_cdi_requirements_from_env still treats any OPENSHELL_CDI_CONTEXT in the supervisor environment as authoritative. Docker now clears/sets it, but this supervisor code is shared with other runtimes and local modes, so an image-provided env plus image-provided files under /run/openshell/supervisor/... can trigger CDI enrichment, including /proc read-write and supplemental groups. Please require driver-attested CDI context: reject empty selected/spec dirs, verify each spec dir is an actual driver-provided mount under CDI_SPEC_DIR_BASE, ensure non-CDI drivers clear the env, and add a regression test for image-supplied OPENSHELL_CDI_CONTEXT.

elezar · 2026-07-22T16:32:23Z

+        let cdi_context = self
+            .config
+            .gpu
+            .cdi_context(gpu_devices.as_deref())


gator-agent

Warning: The selected IDs passed into the CDI context can come from explicit driver_config.cdi_devices, and current validation only checks count/duplicates, not that each ID is in the NVIDIA GPU CDI namespace. Since this PR now derives device, mount, and group policy from those specs, arbitrary CDI vendors can broaden sandbox permissions behind a GPU request. Please require nvidia.com/gpu= for explicit IDs, matching the default inventory filter, and add a negative unit test.

elezar force-pushed the feat/1606-derive-cdi-sandbox-requirements/elezar branch from 61d751d to c8ebad0 Compare July 14, 2026 13:18

elezar force-pushed the feat/1606-derive-cdi-sandbox-requirements/elezar branch 6 times, most recently from d7f06ff to 5cd240a Compare July 20, 2026 11:18

elezar marked this pull request as ready for review July 20, 2026 13:57

elezar requested review from a team, derekwaynecarr, maxamillion and mrunalp as code owners July 20, 2026 13:57

elezar added test:e2e Requires end-to-end coverage test:e2e-gpu Requires GPU end-to-end coverage labels Jul 20, 2026

elezar force-pushed the feat/1606-derive-cdi-sandbox-requirements/elezar branch from 5cd240a to c8b3044 Compare July 21, 2026 12:11

elezar mentioned this pull request Jul 21, 2026

Docker driver: --gpu allocates GPU but driver libraries (libnvidia-ml.so) not visible inside sandbox #2145

Open

elezar added 3 commits July 22, 2026 15:41

refactor(docker): group gpu runtime config

c536b31

Signed-off-by: Evan Lezar <elezar@nvidia.com>

refactor(sandbox): use imported PathBuf consistently

db77b2f

Signed-off-by: Evan Lezar <elezar@nvidia.com>

feat(gpu): derive sandbox policy from CDI specs

c040b8d

Signed-off-by: Evan Lezar <elezar@nvidia.com>

elezar force-pushed the feat/1606-derive-cdi-sandbox-requirements/elezar branch from c8b3044 to c040b8d Compare July 22, 2026 13:42

drew reviewed Jul 22, 2026

View reviewed changes

elezar commented Jul 22, 2026

View reviewed changes

elezar added the gator:in-review Gator is reviewing or awaiting PR review feedback label Jul 22, 2026

fix(gpu): harden CDI context handling

cf6a995

Signed-off-by: Evan Lezar <elezar@nvidia.com>

elezar commented Jul 22, 2026

View reviewed changes

Uh oh!

Conversation

elezar commented Jul 14, 2026

Summary

Related Issue

Changes

Testing

Checklist

Uh oh!

copy-pr-bot Bot commented Jul 14, 2026

Uh oh!

github-actions Bot commented Jul 14, 2026

Uh oh!

github-actions Bot commented Jul 20, 2026

Uh oh!

github-actions Bot commented Jul 20, 2026

Uh oh!

drew Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

elezar left a comment

Choose a reason for hiding this comment

PR Review Status

Uh oh!

elezar Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

elezar Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

elezar Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

elezar Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

elezar left a comment

Choose a reason for hiding this comment

PR Review Status

Uh oh!

elezar Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

elezar Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants