Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]: AzureContainerAppsV1 fails with: Error: Unable to set the Azure CLI to dynamically install missing extensions. #20781

Closed
3 of 7 tasks
KristofKuli opened this issue Jan 13, 2025 · 24 comments
Labels

Comments

@KristofKuli
Copy link

New issue checklist

Task name

AzureContainerAppsV1

Task version

1.247.1

Issue Description

AzureContainerAppsV1 task started to fail with the following error message:
ERROR: The command failed with an unexpected error. Here is the traceback:
ERROR: [Errno 1] Operation not permitted: '/opt/az-config/config'
Traceback (most recent call last):
File "/opt/az/lib/python3.12/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/az/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 666, in execute
raise ex
File "/opt/az/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 733, in _run_jobs_serially
results.append(self._run_job(expanded_arg, cmd_copy))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
##[error]Error Code: [1]
##[error]Error: Unable to set the Azure CLI to dynamically install missing extensions.

Environment type (Please select at least one enviroment where you face this issue)

  • Self-Hosted
  • Microsoft Hosted
  • VMSS Pool
  • Container

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

Ubuntu 22.04

Relevant log output

ERROR: The command failed with an unexpected error. Here is the traceback:
ERROR: [Errno 1] Operation not permitted: '/opt/az-config/config'
Traceback (most recent call last):
  File "/opt/az/lib/python3.12/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.12/site-packages/azure/cli/core/commands/__init__.py", line 666, in execute
    raise ex
  File "/opt/az/lib/python3.12/site-packages/azure/cli/core/commands/__init__.py", line 733, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
##[error]Error Code: [1]
##[error]Error: Unable to set the Azure CLI to dynamically install missing extensions.

Full task logs with system.debug enabled

2025-01-13T09:11:22.4784871Z ##[debug]Evaluating condition for step: 'AzureContainerApps' 2025-01-13T09:11:22.4785650Z ##[debug]Evaluating: SucceededNode() 2025-01-13T09:11:22.4785891Z ##[debug]Evaluating SucceededNode: 2025-01-13T09:11:22.4786273Z ##[debug]=> True 2025-01-13T09:11:22.4786557Z ##[debug]Result: True 2025-01-13T09:11:22.4786812Z ##[section]Starting: AzureContainerApps 2025-01-13T09:11:22.4791286Z ============================================================================== 2025-01-13T09:11:22.4791405Z Task : Azure Container Apps Deploy 2025-01-13T09:11:22.4791485Z Description : An Azure DevOps Task to build and deploy Azure Container Apps. 2025-01-13T09:11:22.4791592Z Version : 1.247.1 2025-01-13T09:11:22.4791659Z Author : Microsoft Corporation 2025-01-13T09:11:22.4791797Z Help : [Learn more about this task](http://github.com/microsoft/azure-pipelines-tasks/blob/master/Tasks/AzureContainerAppsV1/README.md) 2025-01-13T09:11:22.4791926Z ============================================================================== 2025-01-13T09:11:22.5292777Z ##[debug]Using node path: /agent/externals/node20_1/bin/node 2025-01-13T09:11:22.6283336Z ##[debug]system.debug=True 2025-01-13T09:11:22.6287337Z ##[debug]DistributedTask.Tasks.Node.SkipDebugLogsWhenDebugModeOff=True 2025-01-13T09:11:22.6338673Z ##[debug]agent.TempDirectory=/agent/_work/_temp 2025-01-13T09:11:22.6345719Z ##[debug]loading inputs and endpoints 2025-01-13T09:11:22.6351003Z ##[debug]loading INPUT_CWD 2025-01-13T09:11:22.6364635Z ##[debug]loading INPUT_CONNECTEDSERVICENAMEARM 2025-01-13T09:11:22.6367122Z ##[debug]loading INPUT_IMAGETODEPLOY 2025-01-13T09:11:22.6369402Z ##[debug]loading INPUT_CONTAINERAPPNAME 2025-01-13T09:11:22.6371878Z ##[debug]loading INPUT_RESOURCEGROUP 2025-01-13T09:11:22.6374562Z ##[debug]loading ENDPOINT_AUTH_1ae6da43-34f5-4383-90d0-10443c636c13 2025-01-13T09:11:22.6376591Z ##[debug]loading ENDPOINT_AUTH_SCHEME_1ae6da43-34f5-4383-90d0-10443c636c13 2025-01-13T09:11:22.6379971Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_1ae6da43-34f5-4383-90d0-10443c636c13_TENANTID 2025-01-13T09:11:22.6381235Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_1ae6da43-34f5-4383-90d0-10443c636c13_SERVICEPRINCIPALID 2025-01-13T09:11:22.6384984Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_1ae6da43-34f5-4383-90d0-10443c636c13_AUTHENTICATIONTYPE 2025-01-13T09:11:22.6387430Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_1ae6da43-34f5-4383-90d0-10443c636c13_SERVICEPRINCIPALKEY 2025-01-13T09:11:22.6437653Z ##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION 2025-01-13T09:11:22.6438407Z ##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION 2025-01-13T09:11:22.6439195Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN 2025-01-13T09:11:22.6439896Z ##[debug]loading SECRET_SYSTEM_ACCESSTOKEN 2025-01-13T09:11:22.6441027Z ##[debug]loading SECRET_CONTAINER_AUTHENTICATIONTOKEN 2025-01-13T09:11:22.6441679Z ##[debug]loaded 16 2025-01-13T09:11:22.6442452Z ##[debug]Agent.ProxyUrl=http://192.168.1.1:8080 2025-01-13T09:11:22.6443686Z ##[debug]Agent.ProxyUsername=undefined 2025-01-13T09:11:22.6444854Z ##[debug]Agent.ProxyPassword=undefined 2025-01-13T09:11:22.6446674Z ##[debug]Agent.ProxyBypassList= 2025-01-13T09:11:22.6448842Z ##[debug]expose agent proxy configuration. 2025-01-13T09:11:22.6449527Z ##[debug]Agent.CAInfo=undefined 2025-01-13T09:11:22.6450199Z ##[debug]Agent.ClientCert=undefined 2025-01-13T09:11:22.6450886Z ##[debug]Agent.SkipCertValidation=undefined 2025-01-13T09:11:22.7274434Z ##[debug]check path : /agent/_work/_tasks/AzureContainerApps_cad8dd6a-de28-4d89-a7db-1bc20a2fbb2d/1.247.1/node_modules/azure-pipelines-tasks-azure-arm-rest/module.json 2025-01-13T09:11:22.7275933Z ##[debug]adding resource file: /agent/_work/_tasks/AzureContainerApps_cad8dd6a-de28-4d89-a7db-1bc20a2fbb2d/1.247.1/node_modules/azure-pipelines-tasks-azure-arm-rest/module.json 2025-01-13T09:11:22.7278240Z ##[debug]system.culture=en-US 2025-01-13T09:11:22.7296450Z ##[debug]Build.BuildId=11211799 2025-01-13T09:11:22.7297584Z ##[debug]Build.BuildNumber=Deploy DIGIWAGE SLZ Gateway Service 2025-01-13T09:11:22.7299769Z ##[debug]disableTelemetry=undefined 2025-01-13T09:11:22.7303142Z ##[debug]check path : /agent/_work/_tasks/AzureContainerApps_cad8dd6a-de28-4d89-a7db-1bc20a2fbb2d/1.247.1/task.json 2025-01-13T09:11:22.7304640Z ##[debug]adding resource file: /agent/_work/_tasks/AzureContainerApps_cad8dd6a-de28-4d89-a7db-1bc20a2fbb2d/1.247.1/task.json 2025-01-13T09:11:22.7307947Z ##[debug]system.culture=en-US 2025-01-13T09:11:22.7313893Z ##[debug]cwd=/agent/_work/1/s 2025-01-13T09:11:22.7315334Z ##[debug]testing directory '/agent/_work/1/s' 2025-01-13T09:11:22.7337260Z ##[debug]appSourcePath=undefined 2025-01-13T09:11:22.7337968Z ##[debug]acrName=undefined 2025-01-13T09:11:22.7338792Z ##[debug]imageToDeploy=***/digiwagegatewayservice:11211799 2025-01-13T09:11:22.7340364Z ##[debug]yamlConfigPath=undefined 2025-01-13T09:11:22.7341129Z ##[debug]connectedServiceNameARM=1ae6da43-34f5-4383-90d0-10443c636c13 2025-01-13T09:11:22.7345056Z ##[debug]1ae6da43-34f5-4383-90d0-10443c636c13 auth scheme = ServicePrincipal 2025-01-13T09:11:22.7347146Z ##[debug]1ae6da43-34f5-4383-90d0-10443c636c13 auth param authenticationType = spnKey 2025-01-13T09:11:22.7350262Z ##[debug]1ae6da43-34f5-4383-90d0-10443c636c13 auth param serviceprincipalid = *** 2025-01-13T09:11:22.7351076Z ##[debug]1ae6da43-34f5-4383-90d0-10443c636c13 auth param tenantid = 45597f60-6e37-4be7-acfb-4c9e23b261ea 2025-01-13T09:11:22.7352466Z ##[debug]key based endpoint 2025-01-13T09:11:22.7387506Z ##[debug]1ae6da43-34f5-4383-90d0-10443c636c13 auth param serviceprincipalkey = *** 2025-01-13T09:11:22.7391076Z ##[debug]Processed: ##vso[task.setsecret]*** 2025-01-13T09:11:22.7391770Z ##[debug]which 'az' 2025-01-13T09:11:22.7392312Z ##[debug]found: '/usr/bin/az' 2025-01-13T09:11:22.7393115Z ##[debug]/usr/bin/az arg: login --service-principal -u "***" --password=*** --tenant "45597f60-6e37-4be7-acfb-4c9e23b261ea" --allow-no-subscriptions 2025-01-13T09:11:22.7394126Z ##[debug]/usr/bin/az arg: login --service-principal -u "***" --password=*** --tenant "45597f60-6e37-4be7-acfb-4c9e23b261ea" --allow-no-subscriptions 2025-01-13T09:11:22.7394819Z ##[debug]exec tool: /usr/bin/az 2025-01-13T09:11:22.7395394Z ##[debug]exec tool: /usr/bin/az 2025-01-13T09:11:22.7395958Z ##[debug]arguments: 2025-01-13T09:11:22.7396533Z ##[debug]arguments: 2025-01-13T09:11:22.7397060Z ##[debug] login 2025-01-13T09:11:22.7397589Z ##[debug] login 2025-01-13T09:11:22.7398125Z ##[debug] --service-principal 2025-01-13T09:11:22.7398705Z ##[debug] --service-principal 2025-01-13T09:11:22.7399273Z ##[debug] -u 2025-01-13T09:11:22.7399795Z ##[debug] -u 2025-01-13T09:11:22.7400673Z ##[debug] *** 2025-01-13T09:11:22.7401285Z ##[debug] *** 2025-01-13T09:11:22.7401927Z ##[debug] --password=*** 2025-01-13T09:11:22.7402580Z ##[debug] --password=*** 2025-01-13T09:11:22.7403158Z ##[debug] --tenant 2025-01-13T09:11:22.7403765Z ##[debug] --tenant 2025-01-13T09:11:22.7404376Z ##[debug] 45597f60-6e37-4be7-acfb-4c9e23b261ea 2025-01-13T09:11:22.7404988Z ##[debug] 45597f60-6e37-4be7-acfb-4c9e23b261ea 2025-01-13T09:11:22.7405599Z ##[debug] --allow-no-subscriptions 2025-01-13T09:11:22.7406178Z ##[debug] --allow-no-subscriptions 2025-01-13T09:11:22.7407475Z [command]/usr/bin/az login --service-principal -u *** --password=*** --tenant 45597f60-6e37-4be7-acfb-4c9e23b261ea --allow-no-subscriptions 2025-01-13T09:11:24.0601992Z [ 2025-01-13T09:11:24.0776724Z ##[debug]1ae6da43-34f5-4383-90d0-10443c636c13 data SubscriptionID = 53023cb0-9f57-476d-991f-ec705d77d495 2025-01-13T09:11:24.0777446Z ##[debug]which 'az' 2025-01-13T09:11:24.0778062Z ##[debug]found: '/usr/bin/az' 2025-01-13T09:11:24.0778764Z ##[debug]/usr/bin/az arg: account set --subscription "53023cb0-9f57-476d-991f-ec705d77d495" 2025-01-13T09:11:24.0779493Z ##[debug]/usr/bin/az arg: account set --subscription "53023cb0-9f57-476d-991f-ec705d77d495" 2025-01-13T09:11:24.0780135Z ##[debug]exec tool: /usr/bin/az 2025-01-13T09:11:24.0780762Z ##[debug]exec tool: /usr/bin/az 2025-01-13T09:11:24.0781368Z ##[debug]arguments: 2025-01-13T09:11:24.0781989Z ##[debug]arguments: 2025-01-13T09:11:24.0782578Z ##[debug] account 2025-01-13T09:11:24.0783160Z ##[debug] account 2025-01-13T09:11:24.0783774Z ##[debug] set 2025-01-13T09:11:24.0784380Z ##[debug] set 2025-01-13T09:11:24.0784986Z ##[debug] --subscription 2025-01-13T09:11:24.0785553Z ##[debug] --subscription 2025-01-13T09:11:24.0786196Z ##[debug] 53023cb0-9f57-476d-991f-ec705d77d495 2025-01-13T09:11:24.0786886Z ##[debug] 53023cb0-9f57-476d-991f-ec705d77d495 2025-01-13T09:11:24.0787558Z [command]/usr/bin/az account set --subscription 53023cb0-9f57-476d-991f-ec705d77d495 2025-01-13T09:11:24.4094214Z ##[debug]Agent environment resources - Disk: / Available 16965.84 MB out of 74244.74 MB, Memory: Used 1441.00 MB out of 7928.00 MB, CPU: Usage 39.32% 2025-01-13T09:11:24.5108737Z ##[warning]Can't find loc string for key: ErrorInSettingUpSubscription 2025-01-13T09:11:24.5109734Z ##[debug]Processed: ##vso[task.issue type=warning;source=TaskInternal;correlationId=240ec56d-8daf-43e7-b3e3-b2052a00759f;]Can't find loc string for key: ErrorInSettingUpSubscription 2025-01-13T09:11:24.5121569Z ##[debug]1ae6da43-34f5-4383-90d0-10443c636c13 auth scheme = ServicePrincipal 2025-01-13T09:11:24.5122892Z ##[debug]1ae6da43-34f5-4383-90d0-10443c636c13 auth param authenticationType = spnKey 2025-01-13T09:11:24.5123859Z ##[debug]which 'az' 2025-01-13T09:11:24.5124587Z ##[debug]found: '/usr/bin/az' 2025-01-13T09:11:24.5125222Z ##[debug]/usr/bin/az arg: config set extension.use_dynamic_install=yes_without_prompt 2025-01-13T09:11:24.5125904Z ##[debug]/usr/bin/az arg: config set extension.use_dynamic_install=yes_without_prompt 2025-01-13T09:11:24.5163453Z ##[debug]exec tool: /usr/bin/az 2025-01-13T09:11:24.5164111Z ##[debug]exec tool: /usr/bin/az 2025-01-13T09:11:24.5164672Z ##[debug]arguments: 2025-01-13T09:11:24.5165222Z ##[debug]arguments: 2025-01-13T09:11:24.5165752Z ##[debug] config 2025-01-13T09:11:24.5166284Z ##[debug] config 2025-01-13T09:11:24.5166824Z ##[debug] set 2025-01-13T09:11:24.5167381Z ##[debug] set 2025-01-13T09:11:24.5167983Z ##[debug] extension.use_dynamic_install=yes_without_prompt 2025-01-13T09:11:24.5168661Z ##[debug] extension.use_dynamic_install=yes_without_prompt 2025-01-13T09:11:24.5169155Z [command]/usr/bin/az config set extension.use_dynamic_install=yes_without_prompt 2025-01-13T09:11:25.0077154Z WARNING: Command group 'config' is experimental and under development. Reference and support levels: https://aka.ms/CLI_refstatus 2025-01-13T09:11:25.0078541Z ##[error]Error Code: [1] 2025-01-13T09:11:25.0079348Z ##[debug]Processed: ##vso[task.issue type=error;source=TaskInternal;correlationId=240ec56d-8daf-43e7-b3e3-b2052a00759f;]Error Code: [1] 2025-01-13T09:11:25.0081477Z ##[error]Error: Unable to set the Azure CLI to dynamically install missing extensions. 2025-01-13T09:11:25.0082158Z ##[debug]Processed: ##vso[task.issue type=error;source=TaskInternal;correlationId=240ec56d-8daf-43e7-b3e3-b2052a00759f;]Error: Unable to set the Azure CLI to dynamically install missing extensions. 2025-01-13T09:11:25.0083370Z ##[debug]task result: Failed 2025-01-13T09:11:25.0084945Z ##[debug]Processed: ##vso[task.complete result=Failed;] 2025-01-13T09:11:25.0089952Z ERROR: The command failed with an unexpected error. Here is the traceback: 2025-01-13T09:11:25.0090526Z ERROR: [Errno 1] Operation not permitted: '/opt/az-config/config' 2025-01-13T09:11:25.0091107Z Traceback (most recent call last): 2025-01-13T09:11:25.0091664Z ##[debug]Attempting to log out from Azure 2025-01-13T09:11:25.0092275Z File "/opt/az/lib/python3.12/site-packages/knack/cli.py", line 233, in invoke 2025-01-13T09:11:25.0092620Z cmd_result = self.invocation.execute(args) 2025-01-13T09:11:25.0092909Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-13T09:11:25.0093376Z File "/opt/az/lib/python3.12/site-packages/azure/cli/core/commands/__init__.py", line 666, in execute 2025-01-13T09:11:25.0093695Z raise ex 2025-01-13T09:11:25.0094189Z File "/opt/az/lib/python3.12/site-packages/azure/cli/core/commands/__init__.py", line 733, in _run_jobs_serially 2025-01-13T09:11:25.0094593Z results.append(self._run_job(expanded_arg, cmd_copy)) 2025-01-13T09:11:25.0094885Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-13T09:11:25.0095353Z File "/opt/az/lib/python3.12/site-packages/azure/cli/core/commands/__init__.py", line 703, in _run_job 2025-01-13T09:11:25.0095704Z result = cmd_copy(params) 2025-01-13T09:11:25.0095953Z ^^^^^^^^^^^^^^^^ 2025-01-13T09:11:25.0096401Z File "/opt/az/lib/python3.12/site-packages/azure/cli/core/commands/__init__.py", line 336, in __call__ 2025-01-13T09:11:25.0096761Z return self.handler(*args, **kwargs) 2025-01-13T09:11:25.0097035Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-13T09:11:25.0097527Z File "/opt/az/lib/python3.12/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler 2025-01-13T09:11:25.0097898Z return op(**command_args) 2025-01-13T09:11:25.0098172Z ^^^^^^^^^^^^^^^^^^ 2025-01-13T09:11:25.0098838Z File "/opt/az/lib/python3.12/site-packages/azure/cli/command_modules/config/custom.py", line 41, in config_set 2025-01-13T09:11:25.0099252Z cmd.cli_ctx.config.set_value(section, name, _normalize_config_value(value)) 2025-01-13T09:11:25.0102376Z File "/opt/az/lib/python3.12/site-packages/knack/config.py", line 161, in set_value 2025-01-13T09:11:25.0102888Z self._config_file_chain[-1].set_value(section, option, value) 2025-01-13T09:11:25.0103382Z File "/opt/az/lib/python3.12/site-packages/knack/config.py", line 240, in set_value 2025-01-13T09:11:25.0103712Z self.set(config) 2025-01-13T09:11:25.0104135Z File "/opt/az/lib/python3.12/site-packages/knack/config.py", line 229, in set 2025-01-13T09:11:25.0104479Z os.chmod(self.config_path, stat.S_IRUSR | stat.S_IWUSR) 2025-01-13T09:11:25.0104934Z PermissionError: [Errno 1] Operation not permitted: '/opt/az-config/config' 2025-01-13T09:11:25.0105465Z To check existing issues, please visit: https://github.com/Azure/azure-cli/issues 2025-01-13T09:11:25.0106054Z ##[debug]which 'az' 2025-01-13T09:11:25.0106632Z ##[debug]found: '/usr/bin/az' 2025-01-13T09:11:25.0119387Z ##[debug]/usr/bin/az arg: account clear 2025-01-13T09:11:25.0120629Z ##[debug]/usr/bin/az arg: account clear 2025-01-13T09:11:25.0121247Z ##[debug]exec tool: /usr/bin/az 2025-01-13T09:11:25.0121811Z ##[debug]exec tool: /usr/bin/az 2025-01-13T09:11:25.0122380Z ##[debug]arguments: 2025-01-13T09:11:25.0122931Z ##[debug]arguments: 2025-01-13T09:11:25.0123470Z ##[debug] account 2025-01-13T09:11:25.0124046Z ##[debug] account 2025-01-13T09:11:25.0124595Z ##[debug] clear 2025-01-13T09:11:25.0125176Z ##[debug] clear 2025-01-13T09:11:25.0125572Z [command]/usr/bin/az account clear 2025-01-13T09:11:25.5752141Z ##[debug]Telemetry enabled; logging metadata about task result, length and scenario targeted. 2025-01-13T09:11:25.5753372Z ##[debug]which 'docker' 2025-01-13T09:11:25.5754285Z ##[debug]found: '/usr/bin/docker' 2025-01-13T09:11:25.5755534Z ##[debug]/usr/bin/docker arg: run --rm mcr.microsoft.com/oryx/cli:debian-buster-20230207.2 /bin/bash -c "oryx telemetry --event-name 'ContainerAppsPipelinesTaskV1' --processing-time '2844' --property 'result=failed' " 2025-01-13T09:11:25.5756752Z ##[debug]/usr/bin/docker arg: run --rm mcr.microsoft.com/oryx/cli:debian-buster-20230207.2 /bin/bash -c "oryx telemetry --event-name 'ContainerAppsPipelinesTaskV1' --processing-time '2844' --property 'result=failed' " 2025-01-13T09:11:25.5758106Z ##[debug]exec tool: /usr/bin/docker 2025-01-13T09:11:25.5845297Z ##[debug]exec tool: /usr/bin/docker 2025-01-13T09:11:25.5845970Z ##[debug]arguments: 2025-01-13T09:11:25.5846539Z ##[debug]arguments: 2025-01-13T09:11:25.5847103Z ##[debug] run 2025-01-13T09:11:25.5847643Z ##[debug] run 2025-01-13T09:11:25.5848230Z ##[debug] --rm 2025-01-13T09:11:25.5848840Z ##[debug] --rm 2025-01-13T09:11:25.5849498Z ##[debug] mcr.microsoft.com/oryx/cli:debian-buster-20230207.2 2025-01-13T09:11:25.5850243Z ##[debug] mcr.microsoft.com/oryx/cli:debian-buster-20230207.2 2025-01-13T09:11:25.5850942Z ##[debug] /bin/bash 2025-01-13T09:11:25.5851561Z ##[debug] /bin/bash 2025-01-13T09:11:25.5852160Z ##[debug] -c 2025-01-13T09:11:25.5852727Z ##[debug] -c 2025-01-13T09:11:25.5853378Z ##[debug] oryx telemetry --event-name 'ContainerAppsPipelinesTaskV1' --processing-time '2844' --property 'result=failed' 2025-01-13T09:11:25.5854232Z ##[debug] oryx telemetry --event-name 'ContainerAppsPipelinesTaskV1' --processing-time '2844' --property 'result=failed' 2025-01-13T09:11:25.5855381Z [command]/usr/bin/docker run --rm mcr.microsoft.com/oryx/cli:debian-buster-20230207.2 /bin/bash -c oryx telemetry --event-name 'ContainerAppsPipelinesTaskV1' --processing-time '2844' --property 'result=failed' 2025-01-13T09:11:30.5093928Z ##[debug]Agent environment resources - Disk: / Available 16964.94 MB out of 74244.74 MB, Memory: Used 1459.00 MB out of 7928.00 MB, CPU: Usage 73.91% 2025-01-13T09:11:35.5092468Z ##[debug]Agent environment resources - Disk: / Available 16596.50 MB out of 74244.74 MB, Memory: Used 1462.00 MB out of 7928.00 MB, CPU: Usage 73.91% 2025-01-13T09:11:36.6948788Z Unable to find image 'mcr.microsoft.com/oryx/cli:debian-buster-20230207.2' locally 2025-01-13T09:11:36.6949816Z debian-buster-20230207.2: Pulling from oryx/cli 2025-01-13T09:11:36.6950247Z d42a0fb443d7: Pulling fs layer 2025-01-13T09:11:36.6950655Z f390d41539fb: Pulling fs layer 2025-01-13T09:11:36.6951024Z 103b21370b9f: Pulling fs layer 2025-01-13T09:11:36.6951402Z af19115d0008: Pulling fs layer 2025-01-13T09:11:36.6951783Z f373a2bb9dad: Pulling fs layer 2025-01-13T09:11:36.6952154Z f76fd7368bd6: Pulling fs layer 2025-01-13T09:11:36.6952538Z 5aca991e314d: Pulling fs layer 2025-01-13T09:11:36.6952910Z a5586f3fc687: Pulling fs layer 2025-01-13T09:11:36.6953255Z af19115d0008: Waiting 2025-01-13T09:11:36.6953594Z f373a2bb9dad: Waiting 2025-01-13T09:11:36.6953925Z f76fd7368bd6: Waiting 2025-01-13T09:11:36.6954262Z 5aca991e314d: Waiting 2025-01-13T09:11:36.6954591Z a5586f3fc687: Waiting 2025-01-13T09:11:36.6954944Z f390d41539fb: Verifying Checksum 2025-01-13T09:11:36.6955307Z f390d41539fb: Download complete 2025-01-13T09:11:36.6955663Z 103b21370b9f: Download complete 2025-01-13T09:11:36.6956010Z f373a2bb9dad: Verifying Checksum 2025-01-13T09:11:36.6956371Z f373a2bb9dad: Download complete 2025-01-13T09:11:36.6956718Z d42a0fb443d7: Verifying Checksum 2025-01-13T09:11:36.6957077Z d42a0fb443d7: Download complete 2025-01-13T09:11:36.6957432Z f76fd7368bd6: Verifying Checksum 2025-01-13T09:11:36.6957782Z f76fd7368bd6: Download complete 2025-01-13T09:11:36.6958127Z a5586f3fc687: Verifying Checksum 2025-01-13T09:11:36.6958482Z a5586f3fc687: Download complete 2025-01-13T09:11:36.6958830Z d42a0fb443d7: Pull complete 2025-01-13T09:11:36.6959209Z af19115d0008: Verifying Checksum 2025-01-13T09:11:36.6959562Z af19115d0008: Download complete 2025-01-13T09:11:36.6959924Z f390d41539fb: Pull complete 2025-01-13T09:11:36.6960274Z 103b21370b9f: Pull complete 2025-01-13T09:11:36.6964498Z 5aca991e314d: Verifying Checksum 2025-01-13T09:11:36.6964910Z 5aca991e314d: Download complete 2025-01-13T09:11:36.6965270Z af19115d0008: Pull complete 2025-01-13T09:11:36.6965625Z f373a2bb9dad: Pull complete 2025-01-13T09:11:36.6965986Z f76fd7368bd6: Pull complete 2025-01-13T09:11:36.6966327Z 5aca991e314d: Pull complete 2025-01-13T09:11:36.6966667Z a5586f3fc687: Pull complete 2025-01-13T09:11:36.6967329Z Digest: sha256:e6a0561e38ec7b73653067aeea3adec0d303e9c208acbbef76325509a5c2e471 2025-01-13T09:11:36.6967944Z Status: Downloaded newer image for mcr.microsoft.com/oryx/cli:debian-buster-20230207.2 2025-01-13T09:11:36.7111660Z ##[section]Finishing: AzureContainerApps

Repro steps

@github-actions github-actions bot added Area: Release triage Area: ABTT Akvelon Build Tasks Team area of work Task: Bash labels Jan 13, 2025
@mordam
Copy link

mordam commented Jan 14, 2025

Any way to downgrade to avoid this CI-killer??

@singhparveen
Copy link

+1 Same issues started since yesterday

@maxmorrow-rrexpress
Copy link

+1 save issue here. just started for us as well.

@TheManDK
Copy link

+1 also here :(

@padamsondev
Copy link

+1 same issue

@mattkruskamp
Copy link

+1 here as well

@om-anand-gh
Copy link

+1 Issue reproducing on Microsoft Hosted Environment as well.

@Abhishek-Vendorlabs
Copy link

+1 breaking the deployment pipeline

@jacknewberry
Copy link

This is also occurring in Github actions when using azure/container-apps-deploy-action@v1.

Run az config set extension.use_dynamic_install=yes_without_prompt
  az config set extension.use_dynamic_install=yes_without_prompt
  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
  env:
    BASE_URL: https://xxxx
    CA_GH_ACTION_START_MILLISECONDS: xxxx
    CA_GH_ACTION_RESULT_ARG: --property 'result=failed'
WARNING: Command group 'config' is experimental and under development. Reference and support levels: https://aka.ms/CLI_refstatus
ERROR: The command failed with an unexpected error. Here is the traceback:
ERROR: [Errno 1] Operation not permitted: '/opt/az-config/config'
Traceback (most recent call last):
  File "/opt/az/lib/python3.12/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.12/site-packages/azure/cli/core/commands/__init__.py", line 666, in execute
    raise ex
  File "/opt/az/lib/python3.12/site-packages/azure/cli/core/commands/__init__.py", line 733, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.12/site-packages/azure/cli/core/commands/__init__.py", line 703, in _run_job
    result = cmd_copy(params)
             ^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.12/site-packages/azure/cli/core/commands/__init__.py", line 336, in __call__
    return self.handler(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.12/site-packages/azure/cli/core/commands/command_operation.py", line 1[21](https://github.com/healthtech1/sanny/actions/runs/12785706735/job/35641693929#step:7:23), in handler
    return op(**command_args)
           ^^^^^^^^^^^^^^^^^^
  File "/opt/az/lib/python3.12/site-packages/azure/cli/command_modules/config/custom.py", line 41, in config_set
    cmd.cli_ctx.config.set_value(section, name, _normalize_config_value(value))
  File "/opt/az/lib/python3.12/site-packages/knack/config.py", line 161, in set_value
    self._config_file_chain[-1].set_value(section, option, value)
  File "/opt/az/lib/python3.12/site-packages/knack/config.py", line 240, in set_value
    self.set(config)
  File "/opt/az/lib/python3.12/site-packages/knack/config.py", line [22](https://github.com/healthtech1/sanny/actions/runs/12785706735/job/35641693929#step:7:24)9, in set
    os.chmod(self.config_path, stat.S_IRUSR | stat.S_IWUSR)
PermissionError: [Errno 1] Operation not permitted: '/opt/az-config/config'
To check existing issues, please visit: https://github.com/Azure/azure-cli/issues
Error: Process completed with exit code 1.

@rishabh-jain424
Copy link

+1 breaking the pipeline

@rishabh-jain424
Copy link

rishabh-jain424 commented Jan 15, 2025

After updating the action version to v2 in GitHub Action workflow, the job is now running.

  • name: Azure Container Apps Build and Deploy
    uses: Azure/container-apps-deploy-action@v2

@jacknewberry , you can try with v2

@gmathijssen
Copy link

+1 breaking the pipeline

@mordam
Copy link

mordam commented Jan 15, 2025

After updating the action version to v2, the job is now running.

  • name: Azure Container Apps Build and Deploy
    uses: Azure/container-apps-deploy-action@v2

@jacknewberry , you can try with v2

rishabh-jain424

It's not clear where you're configuring this. I tried

- task: AzureContainerApps@2

but this gives

A task is missing. The pipeline references a task called 'AzureContainerApps' with version '2'. This indicates that the specific task version can not be found. (job 'Build_Publish', step ''.)

I tried

- task: Azure/container-apps-deploy-action@v2

but this gives

/azure-pipelines.yml (Line: 354, Col: 31): Task reference must be in the format <NAME>@<MAJOR_VERSION> or <NAME>@<MAJOR_VERSION.MINOR_VERSION.PATCH_VERSION>. For example MyTask@2 or [email protected]. The following task reference format is invalid: 'Azure/container-apps-deploy-action@v2' 7045

(ah i see. haven't had enough coffee :) )

https://github.com/Azure/container-apps-deploy-action

'uses' not 'task' key

however I get this error with that

/azure-pipelines.yml (Line: 354, Col: 25): Unexpected value 'uses' 7049

I now realize you provided a snippet for a github workflow, not for a azure pipeline. Might be worth specifying this in your comment.

@nimro
Copy link

nimro commented Jan 15, 2025

If it helps, this has started happening with the ubuntu-22.04 runner image version 20250113.1.0. Deployments on image version 20250105.1.0 succeed.

@mordam
Copy link

mordam commented Jan 15, 2025

This is a CI-killer. Our entire team is blocked. This must be affecting thousands of businesses.

Please MS roll back to previous build!!

@jsheetzmt
Copy link

Also having issues

@nimro
Copy link

nimro commented Jan 15, 2025

It looks like there were some changes around azure-cli configuration in the latest image: actions/runner-images@releases/ubuntu24/20250105...releases/ubuntu24/20250113#diff-701349f89aba93398253ed2ff24c17bb6fc3f5790872dea878f711f1aad661cc

It now sets AZURE_CONFIG_DIR to /opt/az-config.

WORKAROUND

Manually setting the AZURE_CONFIG_DIR back to the default $HOME/.azure (linux and macos, tested on ubuntu-22.04) or %USERPROFILE%\.azure (windows, untested) seems to resolve the issue.

For example:

- task: AzureContainerApps@1
  displayName: "Deploy image"
  env:
    AZURE_CONFIG_DIR: $HOME/.azure
  inputs:
    ...

@padamsondev
Copy link

Can confirm, workaround posted above solves the issue

@DergachevE
Copy link
Contributor

Hi Everyone,

Sorry for the inconvenience caused by the recent Ubuntu images updates.
I can confirm that the mentioned PR caused the issue:
runner-images: PR 11173

Currently we are rolling back Ubuntu images to the previous version - 20250105.1

@medzymathieuroy
Copy link

Having the same issue with a classic azure pipeline (not YAML) and Azure CLI v2 task. I tried the workaround but somehow it did not apply the env variable when set at the task level. I set AZURE_CONFIG_DIR = $Home/.azure at the pipeline level (just regular pipeline variables) and it did the trick.

@DergachevE
Copy link
Contributor

Hi Everyone,
The Ubuntu images were rolled back to the version 20250105.1
Can you please check if the issue doesn't occur anymore?
/cc @medzymathieuroy @mordam @jsheetzmt

@medzymathieuroy
Copy link

Hi Everyone, The Ubuntu images were rolled back to the version 20250105.1 Can you please check if the issue doesn't occur anymore? /cc @medzymathieuroy @mordam @jsheetzmt

Just ran my pipeline that has an AzureContainerApps@1 task and AzureCLI@2 tasks to deploy a Container App Env and a Container App Job. Ran successfully (I confirm it's using the 20250105.1 image) but i still have @nimro 's workaround in place. I'll try to remove the AZURE_CONFIG_DIR pipeline variable and report back.

@medzymathieuroy
Copy link

@DergachevE I confirm that all of my three azure pipeline tasks (Container App Env deploy, Container App deploy, Container App Job deploy) run successfully on the rolled-back agent after removing @nimro 's AZURE_CONFIG_DIR workaround.

@DergachevE
Copy link
Contributor

Thanks for confirmation. Closing this issue for now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests