centos8平台redis cluster集群搭建(redis5.0.7)

一，规划 redis cluster

1,cluster采用六台redis，3主3从

redis1 ： ip: 172.17.0.2

redis2 ： ip: 172.17.0.3

redis3 ： ip: 172.17.0.4

redis4 ： ip: 172.17.0.5

redis5 ： ip: 172.17.0.6

redis6 ： ip: 172.17.0.7

2,为什么需要6个节点？

redis主从集群最少需要6个节点
master节点至少要3个，slave节点也是3个，

因为一个redis集群要对外提供可用的服务，那么集群中必须要有过半的master节点正常工作。

所以如果想搭建一个能够允许 n 个master节点挂掉的集群，那么就要搭建2n+1个master节点的集群。

2个节点，一个宕掉，剩下的1个不超过1半(1)，集群停止工作

3个节点，一个宕掉，剩下的两个超过1半(1.5)，集群继续工作

4个节点，一个宕掉，剩下的3个超过1半（2个），集群继续工作

4个节点，两个宕掉，剩下的2个不超过1半（2个），集群停止工作

3个节点和4个节点，都是只允许一个节点宕掉，

可见3个是最实际的选择

说明：刘宏缔的架构森林是一个专注架构的博客，地址：https://www.cnblogs.com/architectforest

对应的源码可以访问这里获取： https://github.com/liuhongdi/

说明：作者:刘宏缔邮箱: [email protected]

二，在每台机器上安装redis,操作相同

1，安装wget

[root@redis source]# yum install wget

2,安装gcc

[root@redis source]# yum install gcc

3,安装tcl

[root@redis source]# yum install tcl

4,安装make

[root@redis redis-5.0.7]# yum install make

5,下载redis

[root@redis source]# wget http://download.redis.io/releases/redis-5.0.7.tar.gz

6,解压redis

[root@redis source]# tar -zxvf redis-5.0.7.tar.gz

7,编译:

[root@redis source]# cd redis-5.0.7

[root@redis redis-5.0.7]# make MALLOC=libc

8,测试编译效果:

[root@redis redis-5.0.7]# make test

遇到报错：

You need tcl 8.5 or newer in order to run the Redis test

解决：我们已安装了tcl

make test找不到它是因为缺少which命令，

[root@redis redis-5.0.7]# yum install which

9，再来一次:

[root@redis redis-5.0.7]# make clean

[root@redis redis-5.0.7]# make MALLOC=libc

[root@redis redis-5.0.7]# make test

最终看到以下提示则表示编译无问题

\o/ All tests passed without errors!

10,安装

生成目录

[root@redis soft]# mkdir /usr/local/soft/redis5

[root@redis soft]# mkdir /usr/local/soft/redis5/bin

[root@redis soft]# mkdir /usr/local/soft/redis5/conf

复制文件

[root@redis soft]# cp /usr/local/source/redis-5.0.7/src/redis-cli /usr/local/soft/redis5/bin/

[root@redis soft]# cp /usr/local/source/redis-5.0.7/src/redis-server /usr/local/soft/redis5/bin/

[root@redis soft]# cp /usr/local/source/redis-5.0.7/redis.conf /usr/local/soft/redis5/conf/

三，在每台机器上配置redis,注意有区别的地方:

1，生成目录:

[root@redis soft]# mkdir /data/

[root@redis soft]# mkdir /data/redis

[root@redis soft]# mkdir /data/redis/data

[root@redis soft]# mkdir /data/redis/log

[root@redis soft]# mkdir /data/redis/cluster

说明：data存放rdb或aof

log存放日志

cluster存放cluster的配置文件nodes.conf

2,配置redis.conf

配置以下各项

#数据文件存储目录

dir /data/redis/data/

#日志级别

loglevel notice(使用这个默认值，不用变)

#日志文件

logfile "/data/redis/log/redis.log"

#是否以服务方式运行

daemonize yes

#pid文件

pidfile /var/run/redis_6379.pid(使用这个默认值，不用变)

#是否以cluster方式运行

cluster-enabled yes(此行取消注释即可)

#cluster的配置缓存文件

cluster-config-file /data/redis/cluster/nodes-6379.conf

#连接node的超时时间

cluster-node-timeout 15000(此行取消注释即可)

#访问每台redis的密码

requirepass lhd123

#slave访问master的密码,注意与上一个相同

masterauth lhd123

#是否启用保护模式

protected-mode no

#绑定ip:注意要改成自己当前机器的可访问的ip

#如果是127.0.0.1,则只能从本机访问

bind 172.17.0.2

说明：这里要注意的一点：

绑定ip:注意要改成自己当前机器的可访问的ip

四，给每台机器上的redis生成systemd启动文件,操作相同

1,生成service文件

[root@redis conf]# vi /lib/systemd/system/redis.service

内容:

[Unit]

Description=Redis

After=network.target

[Service]

Type=forking

PIDFile=/var/run/redis_6379.pid

ExecStart=/usr/local/soft/redis5/bin/redis-server /usr/local/soft/redis5/conf/redis.conf

ExecReload=/bin/kill -s HUP $MAINPID

ExecStop=/bin/kill -s QUIT $MAINPID

PrivateTmp=true

[Install]

WantedBy=multi-user.target

重载systemd服务

[root@redis conf]# systemctl daemon-reload

启动redis服务

[root@redis conf]# systemctl start redis

测试：

[root@redis conf]# /usr/local/soft/redis5/bin/redis-cli

127.0.0.1:6379> auth lhd123

OK

127.0.0.1:6379> keys *

(empty list or set)

127.0.0.1:6379> set a aaa

(error) CLUSTERDOWN Hash slot not served

系统提示hash slot还没分配,因为cluster还未创建

五，创建cluster,测试写入,(在任一台机器上操作)

1，创建cluster,在上面创建的任一台机器上操作即可

命令:

[root@redis1 /]# /usr/local/soft/redis5/bin/redis-cli -a lhd123 --cluster create 172.17.0.2:6379 172.17.0.3:6379

172.17.0.4:6379 172.17.0.5:6379 172.17.0.6:6379 172.17.0.7:6379 --cluster-replicas 1

在需要回答下面问题时：输入yes

Can I set the above configuration? (type 'yes' to accept): yes

看一个例子：

#-a: auth 访问密码

#--cluster create :用来创建集群

#--cluster-replicas :为集群中的每个主节点创建一个从节点

[root@redis1 conf]# /usr/local/soft/redis5/bin/redis-cli -a lhd123 --cluster create 172.17.0.2:6379 172.17.0.3:6379

172.17.0.4:6379 172.17.0.5:6379 172.17.0.6:6379 172.17.0.7:6379 --cluster-replicas 1

Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.

>>> Performing hash slots allocation on 6 nodes...

Master[0] -> Slots 0 - 5460

Master[1] -> Slots 5461 - 10922

Master[2] -> Slots 10923 - 16383

Adding replica 172.17.0.6:6379 to 172.17.0.2:6379

Adding replica 172.17.0.7:6379 to 172.17.0.3:6379

Adding replica 172.17.0.5:6379 to 172.17.0.4:6379

M: eb701616b26b5a350605ae3ea5f80e4fc79d84c3 172.17.0.2:6379

   slots:[0-5460] (5461 slots) master

M: 5bdaafe57b1c46f61c5910d3822633a516feb4ae 172.17.0.3:6379

   slots:[5461-10922] (5462 slots) master

M: e024e898a21f3f4051abfb0957046dc4a81ef947 172.17.0.4:6379

   slots:[10923-16383] (5461 slots) master

S: 1ca00d6a680fc1b0d617a46996eaaefc3636fd5a 172.17.0.5:6379

   replicates e024e898a21f3f4051abfb0957046dc4a81ef947

S: 9cd94c491211542dbfae96002489c9b63a5a54e7 172.17.0.6:6379

   replicates eb701616b26b5a350605ae3ea5f80e4fc79d84c3

S: 6f90338cef5af2aa9f0580cd660c80d2ea5fab82 172.17.0.7:6379

   replicates 5bdaafe57b1c46f61c5910d3822633a516feb4ae

Can I set the above configuration? (type 'yes' to accept): yes

>>> Nodes configuration updated

>>> Assign a different config epoch to each node

>>> Sending CLUSTER MEET messages to join the cluster

Waiting for the cluster to join

...

>>> Performing Cluster Check (using node 172.17.0.2:6379)

M: eb701616b26b5a350605ae3ea5f80e4fc79d84c3 172.17.0.2:6379

   slots:[0-5460] (5461 slots) master

   1 additional replica(s)

S: 1ca00d6a680fc1b0d617a46996eaaefc3636fd5a 172.17.0.5:6379

   slots: (0 slots) slave

   replicates e024e898a21f3f4051abfb0957046dc4a81ef947

M: 5bdaafe57b1c46f61c5910d3822633a516feb4ae 172.17.0.3:6379

   slots:[5461-10922] (5462 slots) master

   1 additional replica(s)

M: e024e898a21f3f4051abfb0957046dc4a81ef947 172.17.0.4:6379

   slots:[10923-16383] (5461 slots) master

   1 additional replica(s)

S: 9cd94c491211542dbfae96002489c9b63a5a54e7 172.17.0.6:6379

   slots: (0 slots) slave

   replicates eb701616b26b5a350605ae3ea5f80e4fc79d84c3

S: 6f90338cef5af2aa9f0580cd660c80d2ea5fab82 172.17.0.7:6379

   slots: (0 slots) slave

   replicates 5bdaafe57b1c46f61c5910d3822633a516feb4ae

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

2,测试写入:

# -c: Enable cluster mode,用集群模式访问

# -h: Server hostname (default: 127.0.0.1),指定要访问的主机

[root@redis1 conf]# /usr/local/soft/redis5/bin/redis-cli -a lhd123 -c -h 172.17.0.2

Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.

172.17.0.2:6379> set c bb

-> Redirected to slot [7365] located at 172.17.0.3:6379

OK

172.17.0.3:6379> set d bb

-> Redirected to slot [11298] located at 172.17.0.4:6379

OK

172.17.0.4:6379> get b

-> Redirected to slot [3300] located at 172.17.0.2:6379

"bb"

172.17.0.2:6379> get c

-> Redirected to slot [7365] located at 172.17.0.3:6379

"bb"

172.17.0.3:6379> get d

-> Redirected to slot [11298] located at 172.17.0.4:6379

"bb"

六，查看cluster信息(在任一台机器上操作)

1，查看节点列表：

以集群方式登录redis-cli后查看

172.17.0.3:6379> CLUSTER NODES

9cd94c491211542dbfae96002489c9b63a5a54e7 172.17.0.6:6379@16379 slave eb701616b26b5a350605ae3ea5f80e4fc79d84c3 0 1586861138000 1 connected

5bdaafe57b1c46f61c5910d3822633a516feb4ae 172.17.0.3:6379@16379 myself,master - 0 1586861135000 2 connected 5461-10922

6f90338cef5af2aa9f0580cd660c80d2ea5fab82 172.17.0.7:6379@16379 slave 5bdaafe57b1c46f61c5910d3822633a516feb4ae 0 1586861137000 6 connected

e024e898a21f3f4051abfb0957046dc4a81ef947 172.17.0.4:6379@16379 master - 0 1586861136668 3 connected 10923-16383

1ca00d6a680fc1b0d617a46996eaaefc3636fd5a 172.17.0.5:6379@16379 slave e024e898a21f3f4051abfb0957046dc4a81ef947 0 1586861138674 3 connected

eb701616b26b5a350605ae3ea5f80e4fc79d84c3 172.17.0.2:6379@16379 master - 0 1586861136000 1 connected 0-5460

用 check命令查看

#--cluster check: 列出群集中的机器信息

[root@redis2 /]# /usr/local/soft/redis5/bin/redis-cli --cluster check -a lhd123 172.17.0.3:6379

Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.

172.17.0.3:6379 (5bdaafe5...) -> 2 keys | 5462 slots | 1 slaves.

172.17.0.4:6379 (e024e898...) -> 3 keys | 5461 slots | 1 slaves.

172.17.0.2:6379 (eb701616...) -> 2 keys | 5461 slots | 1 slaves.

[OK] 7 keys in 3 masters.

0.00 keys per slot on average.

>>> Performing Cluster Check (using node 172.17.0.3:6379)

M: 5bdaafe57b1c46f61c5910d3822633a516feb4ae 172.17.0.3:6379

   slots:[5461-10922] (5462 slots) master

   1 additional replica(s)

S: 9cd94c491211542dbfae96002489c9b63a5a54e7 172.17.0.6:6379

   slots: (0 slots) slave

   replicates eb701616b26b5a350605ae3ea5f80e4fc79d84c3

S: 6f90338cef5af2aa9f0580cd660c80d2ea5fab82 172.17.0.7:6379

   slots: (0 slots) slave

   replicates 5bdaafe57b1c46f61c5910d3822633a516feb4ae

M: e024e898a21f3f4051abfb0957046dc4a81ef947 172.17.0.4:6379

   slots:[10923-16383] (5461 slots) master

   1 additional replica(s)

S: 1ca00d6a680fc1b0d617a46996eaaefc3636fd5a 172.17.0.5:6379

   slots: (0 slots) slave

   replicates e024e898a21f3f4051abfb0957046dc4a81ef947

M: eb701616b26b5a350605ae3ea5f80e4fc79d84c3 172.17.0.2:6379

   slots:[0-5460] (5461 slots) master

   1 additional replica(s)

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

2,查看节点的信息:

使用CLUSTER INFO命令

#CLUSTER INFO:cluster的统计信息

172.17.0.3:6379> CLUSTER INFO

cluster_state:ok

cluster_slots_assigned:16384

cluster_slots_ok:16384

cluster_slots_pfail:0

cluster_slots_fail:0

cluster_known_nodes:6

cluster_size:3

cluster_current_epoch:6

cluster_my_epoch:2

cluster_stats_messages_ping_sent:2071

cluster_stats_messages_pong_sent:1919

cluster_stats_messages_meet_sent:1

cluster_stats_messages_sent:3991

cluster_stats_messages_ping_received:1915

cluster_stats_messages_pong_received:2072

cluster_stats_messages_meet_received:4

cluster_stats_messages_received:3991

七，创建redis cluster时遇到的报错或问题：

1,创建cluster时，停在Waiting for the cluster to join不动了

1，16379这个集群管理端口被防火墙做了拦截

2, docker环境中,nodes-6379.conf 文件中redis节点id如果相同也会有这个问题

删除文件后重启

例:

[root@redis1 conf]# rm /data/redis/cluster/nodes-6379.conf

rm: remove regular file '/data/redis/cluster/nodes-6379.conf'? y

2,客户端写入时报错：

(error) MOVED

例子：

172.17.0.2:6379> set a aaa

(error) MOVED 15495 172.17.0.4:6379

解决：

因为启动redis-cli时没有设置集群模式所导致

[root@redis1 conf]# /usr/local/soft/redis5/bin/redis-cli -c -h 172.17.0.2

3，客户端写入时报错：

(error) NOAUTH Authentication required.

例子：

[root@redis1 conf]# /usr/local/soft/redis5/bin/redis-cli -c -h 172.17.0.2

172.17.0.2:6379> auth lhd123

OK

172.17.0.2:6379> set a aaaa

-> Redirected to slot [15495] located at 172.17.0.4:6379

(error) NOAUTH Authentication required.

172.17.0.4:6379> auth "lhd123"

OK

172.17.0.4:6379> set a aaaa

OK

解决：

[root@redis1 conf]# /usr/local/soft/redis5/bin/redis-cli -a lhd123 -c -h 172.17.0.2

八，查看redis版本

[root@redis1 /]# /usr/local/soft/redis5/bin/redis-server --version

Redis server v=5.0.7 sha=00000000:0 malloc=libc bits=64 build=c52ab39fadfc446c

九，查看centos版本

[root@redis1 /]# cat /etc/redhat-release

CentOS Linux release 8.1.1911 (Core)

十，针对redis cluster中node的管理，请移步这一篇:

https://www.cnblogs.com/architectforest/p/12714889.html