Skip to content

az network vnet create intermittently fails with Python 3.14 _ModuleLock deadlock while importing requests.structures #33996

Open
#34201
#33997
Open
az network vnet create intermittently fails with Python 3.14 _ModuleLock deadlock while importing requests.structures#33996
#34201
#33997
Assignees
necusjz
Labels
Auto-AssignAuto assign by botAzure CLI TeamThe command of the issue is owned by Azure CLI teamNetworkaz network vnet/lb/nic/dns/etc...act-quality-productivity-squadcustomer-reportedIssues that are reported by GitHub users external to the Azure organization.questionThe issue doesn't require a change to the product in order to be resolved. Most issues start as that
Milestone
Backlog

Description

@ysksuzuki
Yusuke Suzuki (ysksuzuki)
opened on Aug 27, 2026

Describe the bug

On a GitHub-hosted Ubuntu 24.04 runner, az network vnet create intermittently fails because Python detects a module-lock deadlock while importing requests.structures. The failure occurs with Azure CLI 2.89.1 and its bundled Python 3.14.6. It makes our AKS CI workflow flaky: rerunning the same workflow may succeed.

Related command

az network vnet create \
  --resource-group aks-1 \
  --location westus \
  --name aks-vnet \
  --address-prefixes 192.168.8.0/22 fd00::/48

Errors

ERROR: The command failed with an unexpected error. Here is the traceback:
ERROR: deadlock detected by _ModuleLock('requests.structures') at 140537162870352
Traceback (most recent call last):
  File "/opt/az/lib/python3.14/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/commands/__init__.py", line 677, in execute
    raise ex
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/commands/__init__.py", line 820, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
                   ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/commands/__init__.py", line 800, in _run_job
    result = LongRunningOperation(cmd_copy.cli_ctx, 'Starting {}'.format(cmd_copy.name))(result)
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/commands/__init__.py", line 1168, in __call__
    result = poller.result()
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_poller.py", line 105, in result
    self.wait(timeout)
    ~~~~~~~~~^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/core/tracing/decorator.py", line 119, in wrapper_use_tracer
    return func(*args, **kwargs)
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_poller.py", line 127, in wait
    raise self._exception
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_poller.py", line 80, in _start
    for polling_method in self._polling_generator:
                          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/command_modules/network/aaz/latest/network/vnet/_create.py", line 716, in _execute_operations
    yield self.VirtualNetworksCreateOrUpdate(ctx=self.ctx)()
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_operation.py", line 37, in __init__
    self.client = ctx.get_http_client(self.CLIENT_TYPE)
                  ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_command_ctx.py", line 73, in get_http_client
    credential = self.get_login_credential()
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_command_ctx.py", line 58, in get_login_credential
    credential, _, _ = self._profile.get_login_credentials(
                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        subscription_id=self.subscription_id,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        aux_subscriptions=self.aux_subscriptions,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        aux_tenants=self.aux_tenants
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/_profile.py", line 367, in get_login_credentials
    cred = self._create_credential(account)
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/_profile.py", line 663, in _create_credential
    return identity.get_service_principal_credential(username_or_sp_id)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/auth/identity.py", line 245, in get_service_principal_credential
    return ServicePrincipalCredential(client_id, client_credential, **self._msal_app_kwargs)
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/auth/msal_credentials.py", line 127, in __init__
    self._msal_app = ConfidentialClientApplication(client_id, client_credential=client_credential, **kwargs)
                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/msal/application.py", line 614, in __init__
    import requests  # Lazy load
    ^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/requests/__init__.py", line 150, in <module>
    from . import packages, utils
  File "/opt/az/lib/python3.14/site-packages/requests/utils.py", line 58, in <module>
    from .structures import CaseInsensitiveDict
  File "<frozen importlib._bootstrap>", line 1368, in _find_and_load
  File "<frozen importlib._bootstrap>", line 421, in __enter__
  File "<frozen importlib._bootstrap>", line 346, in acquire
_frozen_importlib._DeadlockError: deadlock detected by _ModuleLock('requests.structures') at 140537162870352
To check existing issues, please visit: https://github.com/Azure/azure-cli/issues

Issue script & Debug output

DEBUG: cli.knack.cli: Command arguments: ['network', 'vnet', 'create', '--resource-group', 'aks-1', '--location', 'westus', '--name', 'aks-vnet', '--address-prefixes', '192.168.8.0/22', 'fd00::/48', '--debug']
DEBUG: cli.knack.cli: __init__ debug log:
Cannot enable color.
DEBUG: cli.knack.cli: Event: Cli.PreExecute []
DEBUG: cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7fdcc63a2e50>, <function OutputProducer.on_global_arguments at 0x7fdcc62cc1a0>, <function CLIQuery.on_global_arguments at 0x7fdcc62fe8d0>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
DEBUG: cli.azure.cli.core: Using packaged command index for profile 'latest'.
DEBUG: cli.azure.cli.core: Found installed extension 'aks-preview' (azext_aks_preview).
DEBUG: cli.azure.cli.core: Blending packaged core index with local extension index.
DEBUG: cli.azure.cli.core: Modules found from index for 'network': ['azure.cli.command_modules.network', 'azure.cli.command_modules.privatedns']
DEBUG: cli.azure.cli.core: Loading command modules...
DEBUG: cli.azure.cli.core: Name                  Load Time    Groups  Commands
DEBUG: cli.azure.cli.core: network                  12.796       121       374
DEBUG: cli.azure.cli.core: privatedns                0.418        14        60
DEBUG: cli.azure.cli.core: Loaded command modules:
DEBUG: cli.azure.cli.core: Total (2)                13.215       135       434
DEBUG: cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_next']
DEBUG: cli.azure.cli.core: Loading extensions:
DEBUG: cli.azure.cli.core: Name                  Load Time    Groups  Commands  Directory
DEBUG: cli.azure.cli.core: Total (0)                 0.000         0         0  
DEBUG: cli.azure.cli.core: Loaded 134 groups, 434 commands.
DEBUG: cli.azure.cli.core: Found a match in the command table.
DEBUG: cli.azure.cli.core: Raw command  : network vnet create
DEBUG: cli.azure.cli.core: Command table: network vnet create
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7fdcc60780f0>]
DEBUG: cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/home/runner/.azure/commands/2026-08-24.15-03-27.network_vnet_create.2722.log'.
INFO: az_command_data_logger: command args: network vnet create --resource-group {} --location {} --name {} --address-prefixes {} {} --debug
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x7fdcc5e7d9b0>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x7fdcc5e7da60>, <function register_global_policy_argument.<locals>.add_global_policy_argument at 0x7fdcc5e9cca0>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x7fdcc5e9cd50>, <function register_upcoming_breaking_change_info.<locals>.update_breaking_change_info at 0x7fdcc5e9ce00>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x7fdcc62cc250>, <function CLIQuery.handle_query_parameter at 0x7fdcc62fe980>, <function register_ids_argument.<locals>.parse_ids_arguments at 0x7fdcc5e9cbf0>]
DEBUG: cli.azure.cli.core.auth.persistence: build_persistence: location='/home/runner/.azure/service_principal_entries.json', encrypt=False
DEBUG: cli.azure.cli.core.auth.persistence: build_persistence: location='/home/runner/.azure/msal_token_cache.json', encrypt=False
DEBUG: cli.azure.cli.core.auth.binary_cache: load: /home/runner/.azure/msal_http_cache.bin
DEBUG: cli.azure.cli.core.azclierror: Traceback (most recent call last):
  File "/opt/az/lib/python3.14/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/commands/__init__.py", line 677, in execute
    raise ex
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/commands/__init__.py", line 820, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
                   ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/commands/__init__.py", line 800, in _run_job
    result = LongRunningOperation(cmd_copy.cli_ctx, 'Starting {}'.format(cmd_copy.name))(result)
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/commands/__init__.py", line 1168, in __call__
    result = poller.result()
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_poller.py", line 105, in result
    self.wait(timeout)
    ~~~~~~~~~^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/core/tracing/decorator.py", line 119, in wrapper_use_tracer
    return func(*args, **kwargs)
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_poller.py", line 127, in wait
    raise self._exception
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_poller.py", line 80, in _start
    for polling_method in self._polling_generator:
                          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/command_modules/network/aaz/latest/network/vnet/_create.py", line 716, in _execute_operations
    yield self.VirtualNetworksCreateOrUpdate(ctx=self.ctx)()
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_operation.py", line 37, in __init__
    self.client = ctx.get_http_client(self.CLIENT_TYPE)
                  ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_command_ctx.py", line 73, in get_http_client
    credential = self.get_login_credential()
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_command_ctx.py", line 58, in get_login_credential
    credential, _, _ = self._profile.get_login_credentials(
                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        subscription_id=self.subscription_id,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        aux_subscriptions=self.aux_subscriptions,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        aux_tenants=self.aux_tenants
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/_profile.py", line 367, in get_login_credentials
    cred = self._create_credential(account)
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/_profile.py", line 663, in _create_credential
    return identity.get_service_principal_credential(username_or_sp_id)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/auth/identity.py", line 245, in get_service_principal_credential
    return ServicePrincipalCredential(client_id, client_credential, **self._msal_app_kwargs)
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/auth/msal_credentials.py", line 127, in __init__
    self._msal_app = ConfidentialClientApplication(client_id, client_credential=client_credential, **kwargs)
                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/msal/application.py", line 614, in __init__
    import requests  # Lazy load
    ^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/requests/__init__.py", line 150, in <module>
    from . import packages, utils
  File "/opt/az/lib/python3.14/site-packages/requests/utils.py", line 58, in <module>
    from .structures import CaseInsensitiveDict
  File "<frozen importlib._bootstrap>", line 1368, in _find_and_load
  File "<frozen importlib._bootstrap>", line 421, in __enter__
  File "<frozen importlib._bootstrap>", line 346, in acquire
_frozen_importlib._DeadlockError: deadlock detected by _ModuleLock('requests.structures') at 140586129370704

ERROR: cli.azure.cli.core.azclierror: The command failed with an unexpected error. Here is the traceback:
ERROR: az_command_data_logger: The command failed with an unexpected error. Here is the traceback:
ERROR: cli.azure.cli.core.azclierror: deadlock detected by _ModuleLock('requests.structures') at 140586129370704
Traceback (most recent call last):
  File "/opt/az/lib/python3.14/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/commands/__init__.py", line 677, in execute
    raise ex
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/commands/__init__.py", line 820, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
                   ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/commands/__init__.py", line 800, in _run_job
    result = LongRunningOperation(cmd_copy.cli_ctx, 'Starting {}'.format(cmd_copy.name))(result)
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/commands/__init__.py", line 1168, in __call__
    result = poller.result()
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_poller.py", line 105, in result
    self.wait(timeout)
    ~~~~~~~~~^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/core/tracing/decorator.py", line 119, in wrapper_use_tracer
    return func(*args, **kwargs)
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_poller.py", line 127, in wait
    raise self._exception
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_poller.py", line 80, in _start
    for polling_method in self._polling_generator:
                          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/command_modules/network/aaz/latest/network/vnet/_create.py", line 716, in _execute_operations
    yield self.VirtualNetworksCreateOrUpdate(ctx=self.ctx)()
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_operation.py", line 37, in __init__
    self.client = ctx.get_http_client(self.CLIENT_TYPE)
                  ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_command_ctx.py", line 73, in get_http_client
    credential = self.get_login_credential()
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_command_ctx.py", line 58, in get_login_credential
    credential, _, _ = self._profile.get_login_credentials(
                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        subscription_id=self.subscription_id,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        aux_subscriptions=self.aux_subscriptions,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        aux_tenants=self.aux_tenants
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/_profile.py", line 367, in get_login_credentials
    cred = self._create_credential(account)
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/_profile.py", line 663, in _create_credential
    return identity.get_service_principal_credential(username_or_sp_id)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/auth/identity.py", line 245, in get_service_principal_credential
    return ServicePrincipalCredential(client_id, client_credential, **self._msal_app_kwargs)
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/auth/msal_credentials.py", line 127, in __init__
    self._msal_app = ConfidentialClientApplication(client_id, client_credential=client_credential, **kwargs)
                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/msal/application.py", line 614, in __init__
    import requests  # Lazy load
    ^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/requests/__init__.py", line 150, in <module>
    from . import packages, utils
  File "/opt/az/lib/python3.14/site-packages/requests/utils.py", line 58, in <module>
    from .structures import CaseInsensitiveDict
  File "<frozen importlib._bootstrap>", line 1368, in _find_and_load
  File "<frozen importlib._bootstrap>", line 421, in __enter__
  File "<frozen importlib._bootstrap>", line 346, in acquire
_frozen_importlib._DeadlockError: deadlock detected by _ModuleLock('requests.structures') at 140586129370704
ERROR: az_command_data_logger: deadlock detected by _ModuleLock('requests.structures') at 140586129370704
Traceback (most recent call last):
  File "/opt/az/lib/python3.14/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/commands/__init__.py", line 677, in execute
    raise ex
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/commands/__init__.py", line 820, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
                   ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/commands/__init__.py", line 800, in _run_job
    result = LongRunningOperation(cmd_copy.cli_ctx, 'Starting {}'.format(cmd_copy.name))(result)
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/commands/__init__.py", line 1168, in __call__
    result = poller.result()
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_poller.py", line 105, in result
    self.wait(timeout)
    ~~~~~~~~~^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/core/tracing/decorator.py", line 119, in wrapper_use_tracer
    return func(*args, **kwargs)
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_poller.py", line 127, in wait
    raise self._exception
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_poller.py", line 80, in _start
    for polling_method in self._polling_generator:
                          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/command_modules/network/aaz/latest/network/vnet/_create.py", line 716, in _execute_operations
    yield self.VirtualNetworksCreateOrUpdate(ctx=self.ctx)()
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_operation.py", line 37, in __init__
    self.client = ctx.get_http_client(self.CLIENT_TYPE)
                  ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_command_ctx.py", line 73, in get_http_client
    credential = self.get_login_credential()
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/aaz/_command_ctx.py", line 58, in get_login_credential
    credential, _, _ = self._profile.get_login_credentials(
                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        subscription_id=self.subscription_id,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        aux_subscriptions=self.aux_subscriptions,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        aux_tenants=self.aux_tenants
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/_profile.py", line 367, in get_login_credentials
    cred = self._create_credential(account)
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/_profile.py", line 663, in _create_credential
    return identity.get_service_principal_credential(username_or_sp_id)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/auth/identity.py", line 245, in get_service_principal_credential
    return ServicePrincipalCredential(client_id, client_credential, **self._msal_app_kwargs)
  File "/opt/az/lib/python3.14/site-packages/azure/cli/core/auth/msal_credentials.py", line 127, in __init__
    self._msal_app = ConfidentialClientApplication(client_id, client_credential=client_credential, **kwargs)
                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/msal/application.py", line 614, in __init__
    import requests  # Lazy load
    ^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.14/site-packages/requests/__init__.py", line 150, in <module>
    from . import packages, utils
  File "/opt/az/lib/python3.14/site-packages/requests/utils.py", line 58, in <module>
    from .structures import CaseInsensitiveDict
  File "<frozen importlib._bootstrap>", line 1368, in _find_and_load
  File "<frozen importlib._bootstrap>", line 421, in __enter__
  File "<frozen importlib._bootstrap>", line 346, in acquire
_frozen_importlib._DeadlockError: deadlock detected by _ModuleLock('requests.structures') at 140586129370704

To check existing issues, please visit: https://github.com/Azure/azure-cli/issues
DEBUG: cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x7fdcc60783b0>]
INFO: az_command_data_logger: exit code: 1
INFO: cli.main: Command ran in 15.098 seconds (init: 0.056, invoke: 15.043)
INFO: telemetry.main: Begin splitting cli events and extra events, total events: 1
INFO: telemetry.main: Finish splitting cli events and extra events, cli events: 1
INFO: telemetry.save: Save telemetry record of length 8385 in cache file under /home/runner/.azure/telemetry/20260824150329209
INFO: telemetry.main: Begin creating telemetry upload process.
INFO: telemetry.process: Creating upload process: "/opt/az/bin/python3 /opt/az/lib/python3.14/site-packages/azure/cli/telemetry/init.py /home/runner/.azure /home/runner/.azure/telemetry/20260824150329209"
INFO: telemetry.process: Return from creating process 2743
INFO: telemetry.main: Finish creating telemetry upload process.

Expected behavior

az network vnet create should create the virtual network successfully without encountering a Python module import deadlock.

Environment Summary

--- GitHub runner image ---
ImageOS=ubuntu24
ImageVersion=20260816.277.1
RUNNER_OS=Linux
RUNNER_ARCH=X64

--- Azure CLI package ---
az executable=/usr/bin/az
azure-cli package=2.89.1-1~noble

--- Azure CLI ---
azure-cli                         2.89.1

core                              2.89.1
telemetry                          1.1.0

Extensions:
azure-devops                       1.0.6

Dependencies:
msal                              1.36.0
azure-mgmt-resource               24.0.0

Python location '/opt/az/bin/python3'
Config directory '/home/runner/.azure'
Extensions directory '/opt/az/azcliextensions'

Python (Linux) 3.14.6 (main, Aug 10 2026, 01:10:26) [GCC 13.3.0]

Legal docs and information: aka.ms/AzureCliLegal


--- Azure CLI Python dependencies ---
azure-core=1.39.0
knack=0.14.0
msal=1.36.0
requests=2.33.0
urllib3=2.7.0

--- Azure CLI extensions ---
Experimental    ExtensionType    Name          Path                                  Preview    Version
--------------  ---------------  ------------  ------------------------------------  ---------  ---------
False           whl              aks-preview   /opt/az/azcliextensions/aks-preview   True       22.0.0b3
False           whl              azure-devops  /opt/az/azcliextensions/azure-devops  False      1.0.6

Additional context

No response

Activity

  1. Yusuke Suzuki (ysksuzuki)
    added
    bugThis issue requires a change to an existing behavior in the product in order to be resolved.
    on Aug 27, 2026
  2. microsoft-github-policy-service
    added
    customer-reportedIssues that are reported by GitHub users external to the Azure organization.
    Networkaz network vnet/lb/nic/dns/etc...
    on Aug 27, 2026
  3. microsoft-github-policy-service
    assigned
    Ethan Yang (necusjz)
    on Aug 27, 2026
  4. microsoft-github-policy-service
    added
    Auto-AssignAuto assign by bot
    on Aug 27, 2026

  5. yonzhan commented on Aug 27, 2026

    @yonzhan
    Yong Zhang (yonzhan)
    on Aug 27, 2026
    Collaborator

    Thank you for opening this issue, we will look into it.

  6. microsoft-github-policy-service
    added
    Azure CLI TeamThe command of the issue is owned by Azure CLI team
    questionThe issue doesn't require a change to the product in order to be resolved. Most issues start as that
    act-quality-productivity-squad
    on Aug 27, 2026

  7. x-engineering-agent commented on Aug 27, 2026

    @x-engineering-agent
    x-engineering-agentbot
    on Aug 27, 2026 – with X Engineering Agent
    Contributor

    Bug Analysis

    On GitHub-hosted Ubuntu 24.04 runners, az network vnet create (and any command that
    creates a login credential) intermittently fails with Azure CLI 2.89.1 bundled with Python 3.14.6:

    ERROR: deadlock detected by _ModuleLock('requests.structures')

    Root cause: azure/cli/core/auth/msal_credentials.py constructs ConfidentialClientApplication,
    which lazily does import requests inside msal/application.py. In Python 3.14, module imports use
    a per-module lock (_ModuleLock) and a lazy import triggered concurrently from more than one thread
    (e.g. one thread importing requests while another thread is already mid-import of a module that
    requests transitively imports, such as requests.utils / requests.structures) can deadlock,
    because CPython 3.14 detects potential lock-ordering cycles between per-module import locks and
    raises _frozen_importlib._DeadlockError instead of blocking forever.

    Azure CLI does not control when msal lazily imports requests, but the CLI already depends on
    requests/msal at startup and can avoid triggering this lazy import from a background/worker
    thread altogether by eagerly importing requests (and msal) up front, in the main thread, during
    CLI process startup — before any command handler or credential creation can run on a separate
    thread. This resolves the deadlock by guaranteeing requests.structures is already fully loaded
    into sys.modules before any lazy import races can occur.

    Suggested fix

    • Add an eager import requests (and, if needed, import msal) near the top of CLI startup
      (e.g. in azure/cli/core/__init__.py or azure/cli/core/auth/identity.py, at module import
      time rather than deferred), so the module is fully initialized on the main thread before any
      concurrent code path can trigger a lazy import of the same module from another thread.
    • Add a regression/unit test in src/azure-cli-core/azure/cli/core/tests/ that imports the auth
      identity module and verifies requests (and its submodules) are present in sys.modules
      immediately after import, guarding against the eager import being removed in the future.
    • Do not modify AAZ-generated files under aaz/<profile>/; this fix belongs in
      src/azure-cli-core/azure/cli/core/auth/ (framework code), not in the network command module.

    Use this EXACT PR title: [Core] Fix #33996: az network vnet create: Eagerly import requests during CLI startup to avoid Python 3.14 module-lock deadlock

    Mandatory Codegen execution protocol

    Before editing implementation files, determine whether the affected Core command is AAZ-generated. Files under aaz/<profile>/ are generated output and must never be patched directly, including by an AI agent. Check out Azure/aaz beside Azure/azure-rest-api-specs, Azure/aaz-dev-tools, and the downstream repository. API-schema defects start in the specification; command naming, grouping, arguments, API-version selection, help, and examples belong in the durable Azure/aaz command model; non-modelable client behavior belongs in a handwritten subclass or wrapper in custom.py, registered from commands.py. Agent Assist creates and promotes the corresponding durable Azure/aaz source pull request before it promotes downstream generated output.

    Follow the Azure CLI repository's Codegen workflow and the aaz-dev setup documentation. Set up the checked-out repositories with azdev setup. Use generate only when importing or redesigning command models from Swagger/TypeSpec. For an existing module whose durable Azure/aaz model has been updated, render that model with regenerate:

    aaz-dev cli regenerate --name Core --cli-path <azure-cli>

# New/imported command model only:
aaz-dev cli generate --spec <specification-name> --module Core

    You MUST actually run the generator; do not merely describe it or imitate its output. If the AAZ/specification checkout, local source change, credentials, or generator is unavailable, stop and report the blocker instead of editing generated files. Inspect _aaz_info provenance and the complete regenerated diff, then run focused azdev style, azdev linter, and azdev test validation. For an extension, also update its version and HISTORY.rst, preserve azext_metadata.json compatibility, and let release automation update src/index.json.

    PR title & description format (required)

    This repo enforces a PR format (guide). Please author the PR exactly as follows or CI's Check the Format of Pull Request Title and Content will fail.

    Use this EXACT PR title (copy verbatim, do not reword):

    [Core] Fix #33996: `az network vnet create`: Eagerly import requests during CLI startup to avoid Python 3.14 module-lock deadlock

    Keep the backticks around the command and the Fix #33996: prefix. You may only adjust the wording after the command (the final summary) if the fix changes; the [Core] prefix, issue link, and backticked command must stay.

    Description — follow the PR template and fill in:

    • Link the issue — start the Description with a closing keyword so the PR auto-links and closes it: Fixes #33996.
    • Related command — the az ... command this affects.
    • Description (mandatory) — why the bug happens, what you changed, and the resulting behavior.
    • Testing Guide — example command(s) showing the fix works.
    • History Notes — leave the title to drive the history note, or add extra lines in the same format (component in brackets + the command in backticks), e.g. [Core] `az <command>`: <note>.
    • Keep the template checklist and tick the items you've satisfied.

  8. x-engineering-agent commented on Aug 27, 2026

    @x-engineering-agent
    x-engineering-agentbot
    on Aug 27, 2026 – with X Engineering Agent
    Contributor

    Started a Copilot task in a0x1ab/azure-cli using claude-sonnet-4.6: https://github.com/a0x1ab/azure-cli/tasks/dbd51d32-858a-485c-bb47-fdcc03920a9d

  9. Aditya Pujara (a0x1ab)
    added a commit that references this issue on Aug 27, 2026

    [Core] Fix Azure#33996: `az network vnet create`: Eagerly import requ…

    d8abdad
  10. Yong Zhang (yonzhan)
    removed
    bugThis issue requires a change to an existing behavior in the product in order to be resolved.
    on Aug 27, 2026
  11. Yong Zhang (yonzhan)
    added this to the Backlog milestone on Aug 27, 2026

  12. yonzhan commented on Aug 27, 2026

    @yonzhan
    Yong Zhang (yonzhan)
    on Aug 27, 2026
    Collaborator

    network

  13. Copilot
    added a commit that references this issue on Aug 27, 2026

    [Core] Fix #33996: `az network vnet create`: Eagerly import requests …

    3eb34f3

  14. martinmine commented on Sep 2, 2026

    @martinmine
    Martin Storø Nyfløtt (martinmine)
    on Sep 2, 2026

    I'm also seeing the same behaviour but with az network nsg rule create with same stack trace.

  15. cnovel commented on Sep 28, 2026

    @cnovel
    Cyril (cnovel)
    on Sep 28, 2026

    We have the same issue with Python 3.14 when calling az lab create.

  16. DPassarelli commented on Oct 6, 2026

    @DPassarelli
    David Passarelli (DPassarelli)
    on Oct 6, 2026

    FWIW, the way we figured out how to get around this issue was to find a Docker image that has AZ CLI with Python 3.12: https://mcr.microsoft.com/en-us/artifact/mar/azure-cli/tag/2.91.0-azurelinux3.0

    Running our code inside that container avoids the deadlock problem. We'll keep an eye on this issue and unpin that specific image after it's resolved.

  17. Jared Thivener (jaredthivener)
    added a commit that references this issue on Oct 7, 2026

    Delete the log workspace with the REST call the CLI bug cannot reach (#…

    1c99926

  18. martinmine commented on Oct 8, 2026

    @martinmine
    Martin Storø Nyfløtt (martinmine)
    on Oct 8, 2026

    Has there been any update on this one?

  19. necusjz commented on Oct 9, 2026

    @necusjz
    Ethan Yang (necusjz)
    on Oct 9, 2026
    Member

    19. We have the same issue with Python 3.14 when calling az lab create.

    could you please provide the complete command? az lab create does not exist.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Auto-AssignAuto assign by botAzure CLI TeamThe command of the issue is owned by Azure CLI teamNetworkaz network vnet/lb/nic/dns/etc...act-quality-productivity-squadcustomer-reportedIssues that are reported by GitHub users external to the Azure organization.questionThe issue doesn't require a change to the product in order to be resolved. Most issues start as that

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions