问题描述
在centos7服务器上，我反复尝试下载容器的所有blob(数据库备份).
-bash-4.2 $ azcopy --version
azcopy 7.3.0-netcore
但是，每次下载时，我都会收到"Transfer FAILED"(传输失败)消息，可以存储从几个文件到1000多个文件的任意位置.该容器有超过90000个Blob.
[2018/11/14 04:31:02.027 + 00:00] [详细]开始传输:https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173945_vm => /mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173945_vm
[2018/11/14 04:31:02.028 + 00:00] [详细]开始传输:https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173946 => /mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173946
[2018/11/14 04:31:02.037 + 00:00] [详细]完成传输:https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173926 => /mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173926
[2018/11/14 04:31:02.037 + 00:00] [详细]开始传输:https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173947 => /mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173947
[2018/11/14 04:31:02.044 + 00:00] [详细]传输失败:https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173922_vm => /mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173922_vm.
[2018/11/14 04:31:02] [错误] https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173922_vm:找不到文件' /mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173922_vm'.
[2018/11/14 04:31:02.045 + 00:00] [详细]开始传输:https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173948 => /mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173948
[2018/11/14 04:31:02.050 + 00:00] [详细]传输失败:https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173923 => /mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173923.
[2018/11/14 04:31:02] [错误] https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173923:找不到文件' /mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173923'.
[2018/11/14 04:31:02.050 + 00:00] [详细]传输失败:https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173924_vm => /mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173924_vm.
[2018/11/14 04:31:02] [错误] https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173924_vm:找不到文件' /mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173924_vm'.
[2018/11/14 04:31:02.050 + 00:00] [详细]开始传输:https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173949 => /mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173949
azcopy --source https://~~~~.net/~~~~-$ {CONTAINER_NAME} -base \
-目的地/mnt/data/~~~~/base/$ {BARMAN_BACKUP} \
-源代码~~~~ \
-递归 \
-安静 \
-冗长的 \
-恢复/var/lib/barman/Microsoft/Azure/AzCopy/Download_Barman_Backup
这是原始的上载. . .
传输的文件总数:924862
转移成功: 924862
转移已跳过: 0
转移失败: 0
经过时间: 00.00:32:22
基础副本到Azure RC = 0
传输的文件总数:916715
转移成功: 915483
转移已跳过: 0
转移失败: 1232
经过时间: 00.00:24:44
在UTC 2018 Tue十一月13日22:29:51结束基础下载
on a centos7 server i'm trying repeatedly to download all blobs of a container (database backup).
-bash-4.2$ azcopy --version
azcopy 7.3.0-netcore
However, with every download I get "Transfer FAILED" on anywhere from a few files to over 1000. The container has over 90000 blobs.
[2018/11/14 04:31:02.027+00:00][VERBOSE] Start transfer: https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173945_vm => /mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173945_vm
[2018/11/14 04:31:02.028+00:00][VERBOSE] Start transfer: https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173946 => /mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173946
[2018/11/14 04:31:02.037+00:00][VERBOSE] Finished transfer: https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173926 => /mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173926
[2018/11/14 04:31:02.037+00:00][VERBOSE] Start transfer: https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173947 => /mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173947
[2018/11/14 04:31:02.044+00:00][VERBOSE] Transfer FAILED: https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173922_vm => /mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173922_vm.
[2018/11/14 04:31:02][ERROR] https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173922_vm: Could not find file '/mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173922_vm'.
[2018/11/14 04:31:02.045+00:00][VERBOSE] Start transfer: https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173948 => /mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173948
[2018/11/14 04:31:02.050+00:00][VERBOSE] Transfer FAILED: https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173923 => /mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173923.
[2018/11/14 04:31:02][ERROR] https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173923: Could not find file '/mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173923'.
[2018/11/14 04:31:02.050+00:00][VERBOSE] Transfer FAILED: https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173924_vm => /mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173924_vm.
[2018/11/14 04:31:02][ERROR] https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173924_vm: Could not find file '/mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173924_vm'.
[2018/11/14 04:31:02.050+00:00][VERBOSE] Start transfer: https://~~~~.blob.core.windows.net/~~~~/data/base/10173597/10173949 => /mnt/data/~~~~/base/20181111T073004/data/base/10173597/10173949
azcopy --source https://~~~~.net/~~~~-${CONTAINER_NAME}-base \
--destination /mnt/data/~~~~/base/${BARMAN_BACKUP} \
--source-key ~~~~ \
--recursive \
--quiet \
--verbose \
--resume /var/lib/barman/Microsoft/Azure/AzCopy/Download_Barman_Backup
This was the original upload . . .
Total files transferred: 924862
Transfer successfully: 924862
Transfer skipped: 0
Transfer failed: 0
Elapsed time: 00.00:32:22
copy of base to Azure RC=0
Total files transferred: 916715
Transfer successfully: 915483
Transfer skipped: 0
Transfer failed: 1232
Elapsed time: 00.00:24:44
Ending base download at Tue Nov 13 22:29:51 UTC 2018
Vm有16个cpu，我尝试使用其他--paralle级的cp
The Vm has 16 cpu's and I tried running with different --paralle-level's
推荐答案
这篇关于容器中所有blob的linux azcopy部分失败，目录消失的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！