Ceph:每个 OSD 的 PG 太多

我使用推荐值配置了 Ceph(使用文档中的公式)。我有 3 个 OSD，我的配置(我已将其放在监视器节点和所有 3 个 OSD 上)包括以下内容:

osd pool default size = 2
osd pool default min size = 1
osd pool default pg num = 150
osd pool default pgp num = 150

当我运行 ceph status 时，我得到:

 health HEALTH_WARN
        too many PGs per OSD (1042 > max 300)

由于两个原因，这令人困惑。第一，因为推荐的公式不满足Ceph。其次，也是最令人费解的是，当我的配置显示 150 时，它说我每个 OSD 有 1042 个 PG。

我究竟做错了什么？

最佳答案

在设置 PG 计数之前，您需要了解 3 件事。
1. OSD数量

ceph osd ls

Sample Output:
 0
 1
 2

 Here Total number of osd is three.

2. 池数 ceph osd pool ls 或 rados lspools

Sample Output:
  rbd
  images
  vms
  volumes
  backups

Here Total number of pool is five.

3. 复制计数

ceph osd dump | grep repli

Sample Output:
 pool 0 'rbd' replicated size 2 min_size 2 crush_ruleset 0 object_hash rjenkins pg_num 64 pgp_num 64 last_change 38 flags hashpspool stripe_width 0
 pool 1 'images' replicated size 2 min_size 2 crush_ruleset 1 object_hash rjenkins pg_num 30 pgp_num 30 last_change 40 flags hashpspool stripe_width 0
 pool 2 'vms' replicated size 2 min_size 2 crush_ruleset 1 object_hash rjenkins pg_num 30 pgp_num 30 last_change 42 flags hashpspool stripe_width 0
 pool 3 'volumes' replicated size 2 min_size 2 crush_ruleset 1 object_hash rjenkins pg_num 30 pgp_num 30 last_change 36 flags hashpspool stripe_width 0
 pool 4 'backups' replicated size 2 min_size 2 crush_ruleset 1 object_hash rjenkins pg_num 30 pgp_num 30 last_change 44 flags hashpspool stripe_width 0

You can see each pool has replication count two.

现在让我们开始计算
计算:
总 PG 计算:

Total PGs = (Total_number_of_OSD * 100) / max_replication_count

This result must be rounded up to the nearest power of 2.

示例:
OSD数量:3
复制数量:2
总 PG = (3 * 100)/2 = 150。150 比 2 的最近幂是 256。
所以最大推荐 PG 是 256
您可以为每个 Pool 设置 PG
每个池的总 PG 计算:

Total PGs = ((Total_number_of_OSD * 100) / max_replication_count) / pool count

This result must be rounded up to the nearest power of 2.

示例:
OSD数量:3
复制数量:2
池数:5
总 PG = ((3 * 100)/2 )/5 = 150/5 = 30 。 30 到 2 的最近幂是 32。
所以每个池的 PG 总数是 32。
2 表的幂:

2^0     1
2^1     2
2^2     4
2^3     8
2^4     16
2^5     32
2^6     64
2^7     128
2^8     256
2^9     512
2^10    1024

有用的命令

ceph osd pool create <pool-name> <pg-number> <pgp-number> - To create a new pool

ceph osd pool get <pool-name> pg_num - To get number of PG in a pool

ceph osd pool get <pool-name> pgp_num - To get number of PGP in a pool

ceph osd pool set <pool-name> pg_num <number> - To increase number of PG in a pool

ceph osd pool set <pool-name> pgp_num <number> - To increase number of PGP in a pool

*usually pg and pgp number is same

关于Ceph:每个 OSD 的 PG 太多，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/40771273/