本文介绍了在S3中每个目录的MAX文件的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

如果我有一百万的图像,会是更好地将它们存储在某个文件夹/子文件夹层次结构或只是甩掉他们都直入一桶(没有任何文件夹)?

If I had a million images, would it be better to store them in some folder/sub-folder hierarchy or just dump them all straight into a bucket (without any folders)?

将为倾倒所有的图像转换成一个层次少斗放缓列表操作?

Would dumping all the images into a hierarchy-less bucket slow down LIST operations?

有没有在动态创建文件夹和子文件夹,并设置其访问控制列表(编程来说)一个显著的开销?

Is there a significant overhead in creating folders and sub folders on the fly and setting up their ACLs (programatically speaking)?

推荐答案

S3不尊重分级命名空间。每个桶只包含对象(连同相关的元数据,访问控制列表等),一个数字键映射。

S3 doesn't respect hierarchical namespaces. Each bucket simply contains a number of mappings from key to object (along with associated metadata, ACLs and so on).

即使你的对象的密钥可能包含一个'/',S3把路径作为一个纯字符串,并把所有对象的平面命名空间。

Even though your object's key might contain a '/', S3 treats the path as a plain string and puts all objects in a flat namespace.

在我的经验中,列表操作确实会(线性)不再作为对象数量的增加,但是这可能是一个症状,增加I / O在Amazon的服务器上必需的,并且沿着电线到客户端。

In my experience, LIST operations do take (linearly) longer as object count increases, but this is probably a symptom of the increased I/O required on the Amazon servers, and down the wire to your client.

然而,查询时间似乎并没有增加与对象计数 - 这是最有可能某种O(1)哈希表的实现他们的结束 - 所以有在同一个桶多个对象应该是一样高性能的小水桶正常使用(即未列出)。

However, lookup times do not seem to increase with object count - it's most probably some sort of O(1) hashtable implementation on their end - so having many objects in the same bucket should be just as performant as small buckets for normal usage (i.e. not LISTs).

对于ACL,助学金可以在水桶和每个单独的对象上设置。由于没有等级,他们是你只有两个选择。显然,设置尽可能多的斗宽的赠款将大量减少您的管理员头痛,如果你有上百万的文件,但要记住,你只能的权限,而不是撤销它们,所以斗宽的补助金应是其所有内容的访问控制列表的最大子集。

As for the ACL, grants can be set on the bucket and on each individual object. As there is no hierarchy, they're your only two options. Obviously, setting as many bucket-wide grants will massively reduce your admin headaches if you have millions of files, but remember you can only grant permissions, not revoke them, so the bucket-wide grants should be the maximal subset of the ACL for all its contents.

我建议拆分成独立的水桶为:

I'd recommend splitting into separate buckets for:

  • 完全不同的内容 - 让一个更健全的体系结构具有不同的图像,声音等数据桶
  • 显著不同的ACL - 如果你能有一个桶接收特定的ACL,或两个水桶与不同的ACL,也没有特定对象的ACL的每个对象,取这两个桶

这篇关于在S3中每个目录的MAX文件的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!

08-04 11:40
查看更多