Repository navigation
[Bug]: Azure provisioning fails with azure-mgmt-compute 38.2.0+ #4333
Description
Steps to reproduce
- Install dstack in a fresh environment, e.g.
uv tool install "dstack[server,azure]==0.22.2". This resolvesazure-mgmt-compute38.4.0. - Configure an
azurebackend. - Run
dstack applyfor any configuration that provisions an Azure VM, e.g. a fleet withnodes: 1.
Actual behaviour
Every Azure offer fails and the instance is terminated with NO_OFFERS (All offers failed). The server logs NoRegisteredProviderFound for each offer (see server logs). This affects all Azure VMs (CPU, CUDA, and A10/GRID), since they all use dstack images from the community gallery.
_get_image_ref looks up the community gallery image only to read its unique_id:
dstack/src/dstack/_internal/core/backends/azure/compute.py
Lines 530 to 547 in 90887d0
| def _get_image_ref( | |
| compute_client: compute_mgmt.ComputeManagementClient, | |
| location: str, | |
| variant: VMImageVariant, | |
| ) -> ImageReference: | |
| if settings.DSTACK_VM_BASE_IMAGE_PREFIX: | |
| # Staging images are not published to the community gallery, so reference directly. | |
| image = compute_client.images.get( | |
| resource_group_name="dstack-resources-westeurope", | |
| image_name=variant.get_image_name(), | |
| ) | |
| return ImageReference(id=image.id) | |
| image = compute_client.community_gallery_images.get( | |
| location=location, | |
| public_gallery_name="dstack-ebac134d-04b9-4c2b-8b6c-ad3e73904aa7", # Gen2 | |
| gallery_image_name=variant.get_image_name(), | |
| ) | |
| return ImageReference(community_gallery_image_id=image.unique_id) |
azure-mgmt-compute 38.2.0 (released 2026-07-20) changed the default API version of community_gallery_images.get from 2025-03-03 to 2025-12-03, and 38.4.0 changed it to 2026-03-03. Azure registers these API versions for galleries but not for locations/communityGalleries, so the lookup fails:
azure-mgmt-compute |
API version | Lookup |
|---|---|---|
| 37.2.0 | 2025-03-03 |
OK |
| 38.1.0 | 2025-03-03 |
OK |
| 38.2.0 | 2025-12-03 |
NoRegisteredProviderFound |
| 38.3.0 | 2025-12-03 |
NoRegisteredProviderFound |
| 38.4.0 | 2026-03-03 |
NoRegisteredProviderFound |
The same call succeeds with 38.4.0 when api_version="2024-03-03" is passed explicitly.
dstack's code hasn't changed: the lookup dates back to #701 and the requirement is azure-mgmt-compute>=29.1.0 with no upper bound. CI also installs the newer SDK (uv.lock isn't committed), but its tests make no real Azure calls, so it doesn't catch this. Any installation that resolved its dependencies after 2026-07-20 is affected, regardless of the dstack version. The server Docker image installs dstack[all]==$VERSION without the lock (docker/server/release/Dockerfile), so images built since then (0.20.29 and later) most likely ship an affected version. I haven't checked a published image.
Expected behaviour
Azure VMs are provisioned regardless of the installed azure-mgmt-compute version.
Possible fix: build the community gallery image ID (/CommunityGalleries/{public_gallery_name}/Images/{image_name}) directly instead of looking it up, and turn an "image not found" error at VM creation into a ComputeError with a clear message (e.g. when the image isn't replicated to the offer's region), so the offer is skipped with a warning instead of a traceback.
dstack version
0.22.2 (also master 90887d0 with azure-mgmt-compute>=38.2.0)
Server logs
ERROR dstack._internal.server.background.pipeline_tasks.instances.cloud_provisioning:200 Got exception when launching Standard_NV6ads_A10_v5 in azure/westeurope
Traceback (most recent call last):
...
File ".../dstack/_internal/core/backends/azure/compute.py", line 187, in create_instance
image_reference=_get_image_ref(
File ".../dstack/_internal/core/backends/azure/compute.py", line 542, in _get_image_ref
...
File ".../azure/mgmt/compute/operations/_operations.py", line 48737, in get
raise HttpResponseError(response=response, model=error, error_format=ARMErrorFormat)
azure.core.exceptions.HttpResponseError: (NoRegisteredProviderFound) No registered resource provider found for location 'westeurope' and API version '2026-03-03' for type 'locations/communityGalleries'. The supported api-versions are '..., 2024-03-03, 2024-07-01, 2024-11-01, 2025-03-03, 2025-04-01, 2025-11-01, 2026-03-01, 2026-04-01'. ...Additional information
Similar to #1987 (azure-mgmt-network 28.0.0).
Activity
Build Azure community gallery image ID instead of looking it up (#4334)