On the azure-cli docker image, 10.13.0 of azcopy is unavailable and so az storage copy fails #30635

hkethi002 · 2025-01-09T21:10:57Z

Describe the bug

the docker images

mcr.microsoft.com/azure-cli:latest
mcr.microsoft.com/azure-cli:2.67.0
and presumably others which use the azure linux operating system, using tdnf as the package manager don't have azcopy version 10.13.0 available and so when trying to run:
az storage copy or any other command that delegates out to azcopy fails.

Related command

docker run mcr.microsoft.com/azure-cli:latest az storage copy \
  --source "https://${storageAccountName}.blob.core.windows.net/${containerName}/" \
  --destination "https://${storageAccountName}.blob.core.windows.net/${containerName}/" \
  --recursive \
  --source-account-name "$storageAccountName" \
  --account-name "$storageAccountName" \
  --auth-mode "key" \
  --account-key "$STORAGE_AZURE_ACCOUNTKEY" \
  --source-account-key "$STORAGE_AZURE_ACCOUNTKEY"

reproduces the issue for me

Errors

WARNING: Positional argument '<EXTRA_OPTIONS>' is experimental and under development. Reference and support levels: https://aka.ms/CLI_refstatus
WARNING: Azcopy not found, installing at /root/.azure/bin/azcopy
ERROR: Connection error while attempting to download azcopy 10.13.0. You could also install the specified azcopy version to /root/.azure/bin manually. (<urlopen error [Errno -2] Name or service not known>)

Issue script & Debug output

DEBUG: cli.knack.cli: Command DEBUG: cli.knack.cli: init Cannot enable color.
DEBUG: cli.knack.cli: Event: DEBUG: cli.knack.cli: Event: DEBUG: cli.knack.cli: Event: DEBUG: cli.azure.cli.core: DEBUG: cli.azure.cli.core: DEBUG: cli.azure.cli.core: DEBUG: cli.azure.cli.core: DEBUG: cli.azure.cli.core: DEBUG: cli.azure.cli.core: DEBUG: cli.azure.cli.core: Name DEBUG: cli.azure.cli.core: acr DEBUG: cli.azure.cli.core: acs DEBUG: cli.azure.cli.core: advisor DEBUG: cli.azure.cli.core: ams DEBUG: cli.azure.cli.core: apim DEBUG: cli.azure.cli.core: appconfig DEBUG: cli.azure.cli.core: appservice DEBUG: cli.azure.cli.core: aro DEBUG: cli.azure.cli.core: backup DEBUG: cli.azure.cli.core: batch DEBUG: cli.azure.cli.core: batchai DEBUG: cli.azure.cli.core: billing DEBUG: cli.azure.cli.core: botservice DEBUG: cli.azure.cli.core: cdn DEBUG: cli.azure.cli.core: cloud DEBUG: cli.azure.cli.core: cognitiveservices DEBUG: cli.azure.cli.core: compute_recommender DEBUG: cli.azure.cli.core: computefleet DEBUG: cli.azure.cli.core: config DEBUG: cli.azure.cli.core: configure DEBUG: cli.azure.cli.core: consumption DEBUG: cli.azure.cli.core: container DEBUG: cli.azure.cli.core: containerapp DEBUG: cli.azure.cli.core: cosmosdb DEBUG: cli.azure.cli.core: databoxedge DEBUG: cli.azure.cli.core: dls DEBUG: cli.azure.cli.core: dms DEBUG: cli.azure.cli.core: eventgrid DEBUG: cli.azure.cli.core: eventhubs DEBUG: cli.azure.cli.core: extension DEBUG: cli.azure.cli.core: feedback DEBUG: cli.azure.cli.core: find DEBUG: cli.azure.cli.core: hdinsight DEBUG: cli.azure.cli.core: identity DEBUG: cli.azure.cli.core: interactive DEBUG: cli.azure.cli.core: iot DEBUG: cli.azure.cli.core: keyvault DEBUG: cli.azure.cli.core: lab DEBUG: cli.azure.cli.core: managedservices DEBUG: cli.azure.cli.core: maps DEBUG: cli.azure.cli.core: marketplaceordering DEBUG: cli.azure.cli.core: monitor DEBUG: cli.azure.cli.core: mysql DEBUG: cli.azure.cli.core: netappfiles DEBUG: cli.azure.cli.core: network DEBUG: cli.azure.cli.core: policyinsights DEBUG: cli.azure.cli.core: privatedns DEBUG: cli.azure.cli.core: profile DEBUG: cli.azure.cli.core: rdbms DEBUG: cli.azure.cli.core: redis DEBUG: cli.azure.cli.core: relay DEBUG: cli.azure.cli.core: resource DEBUG: cli.azure.cli.core: role DEBUG: cli.azure.cli.core: search DEBUG: cli.azure.cli.core: security DEBUG: cli.azure.cli.core: servicebus DEBUG: cli.azure.cli.core: serviceconnector DEBUG: cli.azure.cli.core: servicefabric DEBUG: cli.azure.cli.core: signalr DEBUG: cli.azure.cli.core: sql DEBUG: cli.azure.cli.core: sqlvm DEBUG: cli.azure.cli.core: storage DEBUG: cli.azure.cli.core: synapse DEBUG: cli.azure.cli.core: util DEBUG: cli.azure.cli.core: vm DEBUG: cli.azure.cli.core: Total (65) DEBUG: cli.azure.cli.core: DEBUG: cli.azure.cli.core: DEBUG: cli.knack.cli: Event: DEBUG: cli.azure.cli.core.azlogging: INFO: az_command_data_logger: DEBUG: cli.knack.cli: Event: DEBUG: cli.azure.cli.core.profiles._shared: File "/usr/lib64/az/lib/python3.12 op = getattr(op, part)
^^^^^^^^^^^^^^^^^
AttributeError: module 'azure.mgmt arguments: ['storage', 'copy', '--debug']
debug log:
Cli.PreExecute []
CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0xffff831b5120>, <function OutputProducer.on_global_arguments at 0xffff82f20cc0>, <function CLIQuery.on_global_arguments at 0xffff82f72840>]
CommandInvoker.OnPreCommandTableCreate []
Command index version or cloud profile is invalid or doesn't match the current command.
Command index has been invalidated.
No module found from index for '['storage', 'copy', '--debug']'
Loading all modules and extensions
Discovered command modules: ['acr', 'acs', 'advisor', 'ams', 'apim', 'appconfig', 'appservice', 'aro', 'backup', 'batch', 'batchai', 'billing', 'botservice', 'cdn', 'cloud', 'cognitiveservices', 'compute_recommender', 'computefleet', 'config', 'configure', 'consumption', 'container', 'containerapp', 'cosmosdb', 'databoxedge', 'dls', 'dms', 'eventgrid', 'eventhubs', 'extension', 'feedback', 'find', 'hdinsight', 'identity', 'interactive', 'iot', 'keyvault', 'lab', 'managedservices', 'maps', 'marketplaceordering', 'monitor', 'mysql', 'netappfiles', 'network', 'policyinsights', 'privatedns', 'profile', 'rdbms', 'redis', 'relay', 'resource', 'role', 'search', 'security', 'servicebus', 'serviceconnector', 'servicefabric', 'signalr', 'sql', 'sqlvm', 'storage', 'synapse', 'util', 'vm']
Loading command modules:
Load Time Groups Commands
0.102 36 149
0.008 14 77
0.000 3 6
0.002 22 100
0.002 14 69
0.002 9 47
0.011 79 270
0.004 1 10
0.002 16 60
0.026 34 102
0.001 10 30
0.006 20 53
0.001 12 42
0.071 8 50
0.001 1 7
0.001 10 33
0.001 1 1
0.004 1 1
0.001 2 7
0.000 2 5
0.006 8 9
0.004 2 17
0.045 37 123
0.005 58 199
0.003 5 28
0.001 6 36
0.001 3 22
0.002 25 96
0.003 13 20
0.000 1 7
0.000 1 2
0.000 1 1
0.003 9 42
0.001 2 11
0.000 1 1
0.050 19 82
0.006 20 113
0.015 9 29
0.005 3 8
0.001 5 13
0.001 1 2
0.266 17 61
0.048 16 56
0.019 8 17
0.059 103 338
0.005 9 17
0.008 14 60
0.001 2 8
0.035 50 208
0.001 7 38
0.010 7 8
0.006 52 232
0.001 17 61
0.006 7 19
0.006 47 98
0.049 14 17
0.011 20 315
0.006 27 80
0.001 10 42
0.006 56 215
0.014 4 20
0.018 59 273
0.005 54 246
0.000 3 7
0.020 58 233
1.002 1185 4649
Loaded 1171 groups, 4649 commands.
Updated command index in 0.002 seconds.
CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0xffff81d1ae80>]
metadata file logging enabled - writing logs to '/root/.azure/commands/2025-01-09.21-07-22.storage_copy.9.log'.
command args: storage copy --debug
CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument..add_subscription_parameter at 0xffff819a0360>]
Traceback (most recent call last):
/site-packages/azure/cli/core/profiles/_shared.py", line 656, in _get_attr
.storage.v2023_05_01.models' has no attribute 'ActiveDirectoryPropertiesAccountType'

DEBUG: cli.azure.cli.core.profiles._shared: Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/profiles/_shared.py", line 656, in _get_attr
op = getattr(op, part)
^^^^^^^^^^^^^^^^^
AttributeError: module 'azure.mgmt.storage.v2023_05_01.models' has no attribute 'ListKeyExpand'

DEBUG: cli.azure.cli.core.profiles._shared: Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/profiles/_shared.py", line 656, in _get_attr
op = getattr(op, part)
^^^^^^^^^^^^^^^^^
AttributeError: module 'azure.mgmt.storage.v2023_05_01.models' has no attribute 'CorsRuleAllowedMethodsItem'

DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument..add_ids_arguments at 0xffff819a0400>, <function register_cache_arguments..add_cache_arguments at 0xffff819a0540>, <function register_upcoming_breaking_change_info..update_breaking_change_info at 0xffff819a05e0>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreParseArgs [<function _documentdb_deprecate at 0xffff800d8860>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0xffff82f20d60>, <function CLIQuery.handle_query_parameter at 0xffff82f728e0>, <function register_ids_argument..parse_ids_arguments at 0xffff819a04a0>]
WARNING: Positional argument '<EXTRA_OPTIONS>' is experimental and under development. Reference and support levels: https://aka.ms/CLI_refstatus
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage auth_mode value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage account value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage connection_string value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage account value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage key value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage sas_token value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Cannot generate sas token. unsupported operand type(s) for +: 'NoneType' and 'str'
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage connection_string value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage account value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage key value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage sas_token value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Cannot generate sas token. unsupported operand type(s) for +: 'NoneType' and 'str'
WARNING: cli.azure.cli.command_modules.storage.azcopy.util: Azcopy not found, installing at /root/.azure/bin/azcopy
DEBUG: cli.azure.cli.core.azclierror: Traceback (most recent call last):
File "/usr/lib/python3.12/urllib/request.py", line 1344, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "/usr/lib/python3.12/http/client.py", line 1336, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.12/http/client.py", line 1382, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.12/http/client.py", line 1331, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.12/http/client.py", line 1091, in _send_output
self.send(msg)
File "/usr/lib/python3.12/http/client.py", line 1035, in send
self.connect()
File "/usr/lib/python3.12/http/client.py", line 1470, in connect
super().connect()
File "/usr/lib/python3.12/http/client.py", line 1001, in connect
self.sock = self._create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/socket.py", line 828, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/socket.py", line 963, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/command_modules/storage/azcopy/util.py", line 76, in install_azcopy
_urlretrieve(file_url, install_location)
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/command_modules/storage/azcopy/util.py", line 224, in _urlretrieve
req = urlopen(url)
^^^^^^^^^^^^
File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen
return opener.open(url, data, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/urllib/request.py", line 515, in open
response = self._open(req, data)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/urllib/request.py", line 532, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "/usr/lib/python3.12/urllib/request.py", line 1392, in https_open
return self.do_open(http.client.HTTPSConnection, req,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/urllib/request.py", line 1347, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.12/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 666, in execute
raise ex
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 733, in _run_jobs_serially
results.append(self._run_job(expanded_arg, cmd_copy))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 725, in _run_job
return cmd_copy.exception_handler(ex)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/command_modules/storage/init.py", line 430, in new_handler
first(ex)
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/command_modules/storage/init.py", line 429, in new_handler
raise ex
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 703, in _run_job
result = cmd_copy(params)
^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 336, in call
return self.handler(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
return op(**command_args)
^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/command_modules/storage/operations/azcopy.py", line 15, in storage_copy
azcopy = AzCopy()
^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/command_modules/storage/azcopy/util.py", line 54, in init
self.install_azcopy(install_location)
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/command_modules/storage/azcopy/util.py", line 80, in install_azcopy
raise CLIError('Connection error while attempting to download azcopy {}. You could also install the '
knack.util.CLIError: Connection error while attempting to download azcopy 10.13.0. You could also install the specified azcopy version to /root/.azure/bin manually. (<urlopen error [Errno -2] Name or service not known>)

ERROR: cli.azure.cli.core.azclierror: Connection error while attempting to download azcopy 10.13.0. You could also install the specified azcopy version to /root/.azure/bin manually. (<urlopen error [Errno -2] Name or service not known>)
ERROR: az_command_data_logger: Connection error while attempting to download azcopy 10.13.0. You could also install the specified azcopy version to /root/.azure/bin manually. (<urlopen error [Errno -2] Name or service not known>)
DEBUG: cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0xffff81d1b100>]
INFO: az_command_data_logger: exit code: 1
INFO: cli.main: Command ran in 1.805 seconds (init: 0.482, invoke: 1.323)
DEBUG: cli.azure.cli.core._profile: Failed to load installationId from AzureRmSurvey.json. [Errno 2] No such file or directory: '/root/.azure/AzureRmContextSettings.json'
INFO: telemetry.main: Begin splitting cli events and extra events, total events: 1
INFO: telemetry.client: Accumulated 0 events. Flush the clients.
INFO: telemetry.main: Finish splitting cli events and extra events, cli events: 1
INFO: telemetry.save: Save telemetry record of length 3686 in cache file under /root/.azure/telemetry/20250109210723243
INFO: telemetry.main: Begin creating telemetry upload process.
INFO: telemetry.process: Creating upload process: "/usr/bin/python3.12 /usr/lib/az/lib/python3.12/site-packages/azure/cli/telemetry/init.py /root/.azure /root/.azure/telemetry/20250109210723243"
INFO: telemetry.process: Return from creating process 14
INFO: telemetry.main: Finish creating telemetry upload process.

Expected behavior

it should work

Environment Summary

azure-cli 2.52.0 *

core 2.52.0 *
telemetry 1.1.0

Extensions:
ml 2.20.0

Dependencies:
msal 1.24.0b1
azure-mgmt-resource 23.1.0b2

Python location '/opt/homebrew/Cellar/azure-cli/2.52.0_1/libexec/bin/python'
Extensions directory '/Users/harshakethineni/.azure/cliextensions'

Python (Darwin) 3.10.13 (main, Aug 24 2023, 22:36:46) [Clang 14.0.3 (clang-1403.0.22.14.1)]

Legal docs and information: aka.ms/AzureCliLegal

Additional context

No response

The text was updated successfully, but these errors were encountered:

yonzhan · 2025-01-09T21:11:07Z

Thank you for opening this issue, we will look into it.

hkethi002 · 2025-01-09T21:13:07Z

I think bumping https://github.com/Azure/azure-cli/blob/dev/src/azure-cli/azure/cli/command_modules/storage/azcopy/util.py#L25 to a more recent version would work

rspinabella · 2025-01-09T22:04:46Z

Same thing is happening to me right now!!!! @hkethi002 @yonzhan

Run azure/[email protected]
with:
inlineScript: az storage blob sync -c '$web' -s 'public' --connection-string '***'

azcliversion: latest

Starting script execution via docker image mcr.microsoft.com/azure-cli:latest
WARNING: This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
WARNING: Positional argument '<EXTRA_OPTIONS>' is experimental and under development. Reference and support levels: https://aka.ms/CLI_refstatus
WARNING: Azcopy not found, installing at /root/.azure/bin/azcopy
ERROR: Connection error while attempting to download azcopy 10.13.0. You could also install the specified azcopy version to /root/.azure/bin manually. (<urlopen error [Errno -2] Name or service not known>)
Error: Error: az cli script failed.
cleaning up container...
MICROSOFT_AZURE_CLI_1736451680971_CONTAINER

hkethi002 · 2025-01-09T22:06:27Z

Same thing is happening to me right now!!!! @hkethi002 @yonzhan

I managed to do a workaround by adding tdnf install azcopy -y to the start of my script I'm running in the container if that helps

praiaJoaoMiranda · 2025-01-09T22:26:42Z

Root Cause

Azure deprecated the previously Azure CDN (https://azureedge.net) and the Azure-CLI attempts to download the azcopy from CDN in case of not found locally

The previous address was built dynamically here and is exactly that which is not working anymore - checked doing this:

wget https://azcopyvnext.azureedge.net/release20211027/azcopy_linux_amd64_10.13.0.tar.gz

The azure-cli module install should be updated to use the new url:

https://azcopyvnext-awgzd8g7aagqhzhe.b02.azurefd.net/release20211027/azcopy_{}_{}_{}.{}

You can confirm that approach works by doing this:

wget https://azcopyvnext-awgzd8g7aagqhzhe.b02.azurefd.net/release20211027/azcopy_linux_amd64_10.13.0.tar.gz

@calvinhzy @evelyn-ys - Please open a PR and fix this, it could cause so much damage to many companies using azcopy inside azure-cli.

In this meanwhile, from our developer side, the best we can do is try to install azcopy manually where you need - like @hkethi002 suggest

rspinabella · 2025-01-09T22:37:17Z

@evelyn-ys @calvinhzy Can we please get an update when this is fixed. Im using Azure CLI and Azcopy through a github runner, theres no way for me to manually install it

gonzolively · 2025-01-10T04:45:38Z

This issue has broken some of our build pipelines that use the azure/cli@v2 action item as of around noon CST today. Would be nice to have this bumped in priority as it seems to be a relatively common issue that disrupts many pipelines globally.

ichesa · 2025-01-10T08:02:07Z

This is happening even with the latest az cli version available as of today (2.67.0). This is happening to everyone using az cli that does not yet have a copy of the azcopy tool downloaded locally, and it is impacting automated deployments, as already mentioned by others here in the post.

Please, this is urgent since it cannot be even resolved by bumping your az cli installation to the latest version, so a fix for this is required as soon as possible as the impact is global.

calvinhzy · 2025-01-10T08:18:12Z

Will get a fix out asap to install azcopy using the latest links here https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10?tabs=dnf#download-the-azcopy-portable-binary
In the mean time please manually install Azcopy before running azcopy related commands. Sorry for the issue.

shurick81 · 2025-01-10T14:16:43Z

Here's an example of installing azcopy in a running container

docker run -it --rm mcr.microsoft.com/azure-cli:2.64.0-amd64

tdnf install tar-1.34-3.cm2.x86_64 -y;
# Using curl in mcr.microsoft.com/azure-cli:2.64.0-amd64 and newer
curl -O https://azcopyvnext-awgzd8g7aagqhzhe.b02.azurefd.net/release20211027/azcopy_linux_amd64_10.13.0.tar.gz;
mkdir /root/.azure;
mkdir /root/.azure/bin;
tar -xf azcopy_linux_amd64_10.13.0.tar.gz --strip-components=1 -C /root/.azure/bin;
# now I am able to use az storage copy

akvarpop · 2025-01-10T14:49:03Z

The same problem:

WARNING: This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
WARNING: Positional argument '<EXTRA_OPTIONS>' is experimental and under development. Reference and support levels: https://aka.ms/CLI_refstatus
WARNING: Azcopy not found, installing at /root/.azure/bin/azcopy
ERROR: Connection error while attempting to download azcopy 10.13.0. You could also install the specified azcopy version to /root/.azure/bin manually. (<urlopen error [Errno -2] Name or service not known>)
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: command terminated with exit code 1

shurick81 · 2025-01-10T15:10:41Z

@akvarpop do you mean that you tried installing azcopy it via downloading the gz file like above but still getting error?

akvarpop · 2025-01-10T15:49:10Z

@shurick81 Nope) it's when i run pipelines. I will be waiting Azure fix till Monday) if no i will try your solution
Thanks!

sebader · 2025-01-10T15:53:25Z

@shurick81 you can also just run (assuming you are running in an Azure Linux / Mariner)

tdnf install -y azcopy

This did the trick for us.

ep-ss-2021 · 2025-01-10T19:16:38Z

Will get a fix out asap to install azcopy using the latest links here https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10?tabs=dnf#download-the-azcopy-portable-binary In the mean time please manually install Azcopy before running azcopy related commands. Sorry for the issue.

Hi @calvinhzy , FYI: Please be aware that azcopy versions >= 10.27.0 potentially have memory leak issue ( Azure/azure-storage-azcopy#2855 ).

shurick81 · 2025-01-10T19:21:40Z

@sebader thanks, that is an option. It is also possible to install as old azcopy version as 10.15.0 via this:

tdnf install azcopy-10.15.0

However, I could not find a way to install the same azcopy version as az cli used to install, which is 10.13.0, using tdnf. That is why I had to come up with more complicated solution.

rspinabella · 2025-01-10T19:27:52Z

I asked github copilot to do it for my github actions workflow and it did it correctly on the second try

lqdev · 2025-01-11T03:11:09Z

tdnf azcopy worked for me as well. Here's the snippet when using v1 in GitHub Actions.

- name: Upload to blob storage
  uses: azure/CLI@v1
  with:
    azcliversion: 2.67.0
    # azcopy workadound https://github.com/Azure/azure-cli/issues/30635
    inlineScript: |
        tdnf install -y azcopy;
        # YOUR SCRIPT

calvinhzy · 2025-01-13T03:41:29Z

@ep-ss-2021 good point regarding unstable latest versions, I might still pin it to 10.13.0 and if the user needs a newer version, they can install manually.

worldspawn · 2025-01-13T06:11:21Z

With the net8 runtime image as a base i was able to use this:

RUN curl -L https://aka.ms/downloadazcopy-v10-linux -o downloadazcopy-v10-linux
RUN tar -xvf downloadazcopy-v10-linux

# Move AzCopy
RUN  rm -f /usr/bin/azcopy
RUN  cp ./azcopy_linux_amd64_*/azcopy /usr/bin/
RUN  chmod 755 /usr/bin/azcopy

# Clean the kitchen
RUN rm -f downloadazcopy-v10-linux
RUN rm -rf ./azcopy_linux_amd64_*/

…of Edgio endpoint for the installation link which has been deprecated (#30640) * update to latest azcopy download link with version 10.27.1 * latest version of azcopy has some memory leak issue, pin back to 10.13.0, if user requires a newer version, can follow the guide here https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10?tabs=dnf#download-the-azcopy-portable-binary

calvinhzy · 2025-01-13T06:31:09Z

In this PR #30640, changed the link to the Azure FrontDoor endpoint like this
https://azcopyvnext-awgzd8g7aagqhzhe.b02.azurefd.net/release20211027/azcopy_windows_amd64_10.13.0.zip
https://azcopyvnext-awgzd8g7aagqhzhe.b02.azurefd.net/release20211027/azcopy_windows_386_10.13.0.zip
https://azcopyvnext-awgzd8g7aagqhzhe.b02.azurefd.net/release20211027/azcopy_linux_amd64_10.13.0.tar.gz
https://azcopyvnext-awgzd8g7aagqhzhe.b02.azurefd.net/release20211027/azcopy_darwin_amd64_10.13.0.zip

The next release to have these changes will be 2/11. So please try some of the solutions provided above to manually install AzCopy depending on the version needed.

AHJiapengZhang · 2025-01-13T10:33:16Z

In this PR #30640, changed the link to the Azure FrontDoor endpoint like this https://azcopyvnext-awgzd8g7aagqhzhe.b02.azurefd.net/release20211027/azcopy_windows_amd64_10.13.0.zip https://azcopyvnext-awgzd8g7aagqhzhe.b02.azurefd.net/release20211027/azcopy_windows_386_10.13.0.zip https://azcopyvnext-awgzd8g7aagqhzhe.b02.azurefd.net/release20211027/azcopy_linux_amd64_10.13.0.tar.gz https://azcopyvnext-awgzd8g7aagqhzhe.b02.azurefd.net/release20211027/azcopy_darwin_amd64_10.13.0.zip

The next release to have these changes will be 2/11. So please try some of the solutions provided above to manually install AzCopy depending on the version needed.

@calvinhzy Could you inform us what is the version number for the next release?

yonzhan · 2025-01-13T11:04:58Z

Official Release: 02/11/2025
Azure CLI version: 2.69.0

camposdelima · 2025-01-13T14:44:52Z

Official Release: 02/11/2025
Azure CLI version: 2.69.0

Official Release: 02/11/2025 ( Feb/11/2025 )
Azure CLI version: 2.69.0

cg505 · 2025-01-16T00:45:34Z

Unless I'm misunderstanding, this affects all usages of azure-cli that don't already have azcopy, not just the azure-cli docker image. E.g. my laptop with a freshly-installed copy of azure-cli. The mitigation steps are unclear. (Where should azcopy be downloaded to? I am not going to pollute my /usr/bin with random binaries to work around this bug.)
A patch version release, ideally for old minor versions as well, would be greatly appreciated.

Azure/azure-cli#30635 (comment)

hkethi002 added the bug This issue requires a change to an existing behavior in the product in order to be resolved. label Jan 9, 2025

microsoft-github-policy-service bot added customer-reported Issues that are reported by GitHub users external to the Azure organization. Storage az storage labels Jan 9, 2025

microsoft-github-policy-service bot assigned evelyn-ys and calvinhzy Jan 9, 2025

microsoft-github-policy-service bot added Auto-Assign Auto assign by bot Azure CLI Team The command of the issue is owned by Azure CLI team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Jan 9, 2025

yonzhan removed the bug This issue requires a change to an existing behavior in the product in order to be resolved. label Jan 10, 2025

yonzhan added this to the Backlog milestone Jan 10, 2025

Jmorjsm mentioned this issue Jan 10, 2025

azcopyvnextrelease Authentication enabled Azure/azure-storage-azcopy#2908

Open

calvinhzy mentioned this issue Jan 10, 2025

{Storage} Fix #30635: az storage copy: Use Azure FrontDoor instead of Edgio endpoint for the installation link which has been deprecated #30640

Merged

3 tasks

calvinhzy closed this as completed in #30640 Jan 13, 2025

crohrer mentioned this issue Jan 14, 2025

Unable to download azcopy 10.13 binary in an Azure Devops Pipelines with custom linux agents Azure/azure-storage-azcopy#1933

Closed

ShaneParra mentioned this issue Jan 14, 2025

ERROR: Connection error while attempting to download azcopy 10.13.0 bacongobbler/azure-blob-storage-upload#39

Open

JoshuaGleason mentioned this issue Jan 15, 2025

BEL-4757 Add new workflows for interactives StrongMind/public-reusable-workflows#227

Merged

cg505 mentioned this issue Jan 16, 2025

upload to Azure storage broken due to azure-cli bug skypilot-org/skypilot#4568

Open

camposdelima added a commit to camposdelima/azure-blob-storage-upload that referenced this issue Jan 17, 2025

Workaround for issue Azure/azure-cli#30635

6cf24f7

Azure/azure-cli#30635 (comment)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

On the azure-cli docker image, 10.13.0 of azcopy is unavailable and so az storage copy fails #30635

On the azure-cli docker image, 10.13.0 of azcopy is unavailable and so az storage copy fails #30635

hkethi002 commented Jan 9, 2025

yonzhan commented Jan 9, 2025

hkethi002 commented Jan 9, 2025

rspinabella commented Jan 9, 2025 •

edited

Loading

hkethi002 commented Jan 9, 2025

praiaJoaoMiranda commented Jan 9, 2025 •

edited

Loading

rspinabella commented Jan 9, 2025

gonzolively commented Jan 10, 2025

ichesa commented Jan 10, 2025

calvinhzy commented Jan 10, 2025

shurick81 commented Jan 10, 2025

akvarpop commented Jan 10, 2025

shurick81 commented Jan 10, 2025 •

edited

Loading

akvarpop commented Jan 10, 2025

sebader commented Jan 10, 2025 •

edited

Loading

ep-ss-2021 commented Jan 10, 2025

shurick81 commented Jan 10, 2025 •

edited

Loading

rspinabella commented Jan 10, 2025

lqdev commented Jan 11, 2025

calvinhzy commented Jan 13, 2025

worldspawn commented Jan 13, 2025

calvinhzy commented Jan 13, 2025

AHJiapengZhang commented Jan 13, 2025

yonzhan commented Jan 13, 2025

camposdelima commented Jan 13, 2025

cg505 commented Jan 16, 2025

On the azure-cli docker image, 10.13.0 of azcopy is unavailable and so az storage copy fails #30635

On the azure-cli docker image, 10.13.0 of azcopy is unavailable and so az storage copy fails #30635

Comments

hkethi002 commented Jan 9, 2025

Describe the bug

Related command

Errors

Issue script & Debug output

Expected behavior

Environment Summary

Additional context

yonzhan commented Jan 9, 2025

hkethi002 commented Jan 9, 2025

rspinabella commented Jan 9, 2025 • edited Loading

hkethi002 commented Jan 9, 2025

praiaJoaoMiranda commented Jan 9, 2025 • edited Loading

Root Cause

rspinabella commented Jan 9, 2025

gonzolively commented Jan 10, 2025

ichesa commented Jan 10, 2025

calvinhzy commented Jan 10, 2025

shurick81 commented Jan 10, 2025

akvarpop commented Jan 10, 2025

shurick81 commented Jan 10, 2025 • edited Loading

akvarpop commented Jan 10, 2025

sebader commented Jan 10, 2025 • edited Loading

ep-ss-2021 commented Jan 10, 2025

shurick81 commented Jan 10, 2025 • edited Loading

rspinabella commented Jan 10, 2025

lqdev commented Jan 11, 2025

calvinhzy commented Jan 13, 2025

worldspawn commented Jan 13, 2025

calvinhzy commented Jan 13, 2025

AHJiapengZhang commented Jan 13, 2025

yonzhan commented Jan 13, 2025

camposdelima commented Jan 13, 2025

cg505 commented Jan 16, 2025

rspinabella commented Jan 9, 2025 •

edited

Loading

praiaJoaoMiranda commented Jan 9, 2025 •

edited

Loading

shurick81 commented Jan 10, 2025 •

edited

Loading

sebader commented Jan 10, 2025 •

edited

Loading

shurick81 commented Jan 10, 2025 •

edited

Loading