问题描述
我们有一个问题，其中s3目录中的某些文件在〜500MiB范围内，而许多其他文件在KiB和Bytes中.我想将所有的小文件合并到数量较少的大文件中，约500MiB.
We have a problem wherein some of the files in a s3 directory are in ~500MiB range, but many other files are in KiB and Bytes. I want to merge all the small files into fewer bigger files of the order of ~500MiB.
在s3文件夹中重写数据而不是下载，在本地合并并推回s3的最有效方法是什么.我可以使用一些实用程序/aws命令来实现它吗?
What is the most efficient way to rewriting data in an s3 folder instead of having to download, merge on local and push back to s3. Is there some utility/aws command i can use to achieve it?
推荐答案
S3是一种存储服务，没有计算功能.对于您的要求，您需要计算(合并).因此，如果不下载，合并和上传，就无法做自己想要做的事情.
S3 is a storage service and has no compute capability. For what you are asking, you need compute (to merge). So you cannot do what you want without downloading, merging and uploading.
这篇关于将s3上的多个zip文件合并为更少的zip文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！