Epic: Move service selection into the backend API instead of pre-filtering the project in the CLI #14346

"This issue was filed by an AI agent on a human's behalf. The human submitter may not have independently verified the report."

Summary

Service selection (up web, --no-deps, watch rebuilds, …) is currently applied by the callers of the backend, by pre-filtering the types.Project with WithSelectedServices. This issue proposes making the selection an explicit input of the backend API, so that the reconciler sees the whole model and decides what the selection means. It also matters for other consumers of pkg/api (library users, Docker Desktop, …): they should not have to reproduce the CLI's pre-filtering to get correct behavior.

Related: #14081 (converging the start phase into the plan engine), #14291.

Problem

Two different notions of "selection" coexist today:

1. Services in the API options (CreateOptions.Services, …) means targeted services. The reconciler still converges every other service, with the RecreateDependencies policy (reconcileService).
2. Pre-filtering the project (project.WithSelectedServices(...), done in cmd/compose/up.go and cmd/compose/scale.go for --no-deps, and in watch's rebuild) means the other services do not exist: they are moved to DisabledServices, and DependsOn of the kept services is pruned to the selection.

Because the reconciler receives a truncated model without knowing it, it makes wrong inferences. Concretely, it looks for the consumers of a network or volume in project.Services only (servicesUsingNetwork, servicesUsingVolume). With a narrowed project, a drifted network or volume that is also used by a service outside the selection looks unused by it:

• the plan stops/disconnects (or removes) only the selected services' containers, then tries to remove the network/volume, which fails with "in use";
• for a volume, the user is first asked to confirm a data-destroying recreation that cannot succeed.

I reproduced this at the reconciler level with unit tests, on a project narrowed to one service whose network (or volume) is shared with a disabled service that has a live container. up --no-deps <svc> and scale --no-deps build exactly such a project, so I expect the same exposure there; I have not reproduced it end to end against a daemon. Dependency pruning in WithSelectedServices may also affect config hashes; I have not checked that.

A narrow guard in the reconciler (leave a drifted resource alone, with a warning, while a disabled service's live container still holds it) is being added to #14291 for the watch rebuild. It is a mitigation: it works around the truncated model rather than removing the cause.

Proposal

Keep the full project as the model and pass the selection to the backend as data. A sketch, open for discussion:

// ServiceSelection scopes an operation to part of the project.
type ServiceSelection struct {
	Services     []string
	Dependencies DependencyScope // include (default), ignore (--no-deps), dependents (watch rebuild)
}

carried by the options of the operations that converge or start containers (CreateOptions, UpOptions/StartOptions, ScaleOptions, WatchOptions), and applied at a single point, in the plan preparation, once the canonical project has been built.

Semantics the reconciler would own:

• The model stays whole: every consumer of a network or volume is visible, and the dependency graph is the real one.
• The selection bounds which services' containers may be created, recreated, started, stopped or waited on.
• Infrastructure needed by the selected services is created if missing. A drifted network or volume is recreated only if all its consumers are within the selection; otherwise it is kept, with a warning.
• Orphan detection is unaffected by the selection.

Compatibility

• Additive: a caller that keeps pre-filtering the project still works, since it just yields a project with fewer services.
• The existing Services fields keep their current meaning ("targets"); the new field must not silently redefine them.
• Commands that never go through the plan (config, pull, push, run's own container) can keep pre-filtering client side. Migration can go command by command, starting with up, create, start, scale and watch.

Questions

• Is a single ServiceSelection value the right shape, or should the dependency scope stay a separate option?
• Should RecreateDependencies be folded into the selection, since it exists to express "what happens to services outside the targets"?
• Are there library consumers relying on the current pre-filtering contract that this would break?

Feedback from anyone consuming pkg/api is welcome.