使用keepalived配置支持Linux集群高可用

Linux集群概述

• 根据功能划分为两大类：高可用和负载均衡
• 高可用集群通常为两台服务器，一台工作，另外一台作为冗余，当提供服务的机器宕机，冗余将接替继续提供服务
• 实现高可用的开源软件有：heartbeat、keepalived，推荐使用keepalived
• 负载均衡集群，需要有一台服务器作为分发器，它负责把用户的请求分发给后端的服务器处理，在这个集群里，除了分发器外，就是给用户提供服务的服务器了，这些服务器数量至少为2
• 实现负载均衡的开源软件有LVS、keepalived、haproxy、nginx，商业的有F5、Netscaler

keepalived介绍

• keepalived通过VRRP（Virtual Router Redundancy Protocol,虚拟路由冗余协议）来实现高可用。
• 在这个协议里会将多台功能相同的路由器组成一个小组，这个小组里会有1个master角色和N（N>=1）个backup角色。
• master会通过组播的形式向各个backup发送VRRP协议的数据包，当backup收不到master发来的VRRP数据包时，就会认为master宕机了。此时就需要根据各个backup的优先级来决定谁成为新的mater
• keepalived要有三个模块，分别是core、check和vrrp。其中core模块为keepalived的核心，负责主进程的启动、维护以及全局配置文件的加载和解析，check模块负责健康检查，vrrp模块是来实现VRRP协议的。

针对Nginx服务进行高可用配置

virtual_ipaddress 绑定的ip, 下面简写vip

安装keepalive
master(test-a:134)与backup(centos0: 129)安装keepalived

[root@test-a ~]# yum install -y keepalived
[root@centos0 ~]# yum install -y keepalived

# 使用yum安装Nginx如果提示找不到对应的安装包，需要安装epel源
[root@centos0 ~]# yum install -y nginx
已加载插件：fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.neusoft.edu.cn
 * updates: mirrors.cqu.edu.cn
没有可用软件包 nginx。
错误：无须任何处理
[root@centos0 ~]# yum install -y nginx # 再安装

master配置启动服务

# 先关闭防火墙
[root@test-a ~]# systemctl stop firewalld.service
[root@test-a ~]# setenforce 0

# 编辑配置文件
[root@test-a ~]# vim /etc/keepalived/keepalived.conf
[root@test-a ~]# cat  /etc/keepalived/keepalived.conf
global_defs {
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}

vrrp_script chk_nginx {
    script "/usr/local/sbin/check_ng.sh"
    interval 3
}

vrrp_instance VI_1 {
    state MASTER
    interface eno16777736
    virtual_router_id 1
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass test111
    }
    virtual_ipaddress {
        192.168.77.100
    }

    track_script {
        chk_nginx
    }

}


[root@test-a ~]# vim /usr/local/sbin/check_ng.sh  # 编写对应的检测脚本
[root@test-a ~]# cat /usr/local/sbin/check_ng.sh
#!/bin/bash
#时间变量，用于记录日志
d=`date --date today +%Y%m%d_%H:%M:%S`
#计算nginx进程数量
n=`ps -C nginx --no-heading|wc -l`
#如果进程为0，则启动nginx，并且再次检测nginx进程数量，
#如果还为0，说明nginx无法启动，此时需要关闭keepalived
echo "$d nginx check begin"
if [ $n -eq "0" ]; then
        /etc/init.d/nginx start
        n2=`ps -C nginx --no-heading|wc -l`
        if [ $n2 -eq "0"  ]; then
                echo "$d nginx down,keepalived will stop" >> /var/log/check_ng.log
                systemctl stop keepalived
        fi
        echo "$d nginx restart ok" >> /var/log/check_ng.log
fi

[root@test-a ~]# chmod 755 /usr/local/sbin/check_ng.sh
[root@test-a ~]# systemctl start keepalived.service
[root@test-a ~]# ps aux|grep keep
root      2841  0.0  0.1 119248  1396 ?        Ss   15:13   0:00 /usr/sbin/keepa                                          lived -D
root      2842  0.0  0.3 123448  3136 ?        S    15:13   0:00 /usr/sbin/keepa                                          lived -D
root      2843  0.1  0.2 123516  2532 ?        S    15:13   0:00 /usr/sbin/keepa                                          lived -D
root      2861  0.0  0.0 112704   972 pts/0    R+   15:14   0:00 grep --color=au                                          to keep
[root@test-a ~]# ps aux|grep nginx
root      1688  0.0  0.1  46548  1268 ?        Ss   15:00   0:00 nginx: master p                                          rocess /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf
nobody    1694  0.0  0.3  49036  3900 ?        S    15:00   0:00 nginx: worker p                                          rocess
nobody    1695  0.0  0.3  49036  3900 ?        S    15:00   0:00 nginx: worker p                                          rocess
root      2911  0.0  0.0 112704   976 pts/0    R+   15:14   0:00 grep --color=au                                          to nginx

[root@test-a ~]# ip add  # 查看绑定的vip地址,ifconfig看不见
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast stat                                          e UP qlen 1000
    link/ether 00:0c:29:b9:56:99 brd ff:ff:ff:ff:ff:ff
    inet 192.168.77.134/24 brd 192.168.77.255 scope global eno16777736
       valid_lft forever preferred_lft forever
    inet 192.168.77.100/32 scope global eno16777736
       valid_lft forever preferred_lft forever
    inet 192.168.77.139/24 brd 192.168.77.255 scope global secondary eno16777736                                          :t
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:feb9:5699/64 scope link
       valid_lft forever preferred_lft forever

[root@test-a ~]# /etc/init.d/nginx stop # nginx停止后，检测脚本会重新启动
Stopping nginx (via systemctl):                            [  OK  ]
[root@test-a ~]# ps aux|grep keep
root      4508  0.0  0.1 119248  1396 ?        Ss   15:21   0:00 /usr/sbin/keepa                                          lived -D
root      4509  0.0  0.3 123448  3120 ?        S    15:21   0:00 /usr/sbin/keepa                                          lived -D
root      4510  0.0  0.2 123516  2652 ?        S    15:21   0:00 /usr/sbin/keepa                                          lived -D
root      5402  0.0  0.0 112704   976 pts/0    R+   15:25   0:00 grep --color=au                                          to keep
[root@test-a ~]# ps aux|grep nginx
root      5388  0.0  0.1  46548  1264 ?        Ss   15:25   0:00 nginx: master p                                          rocess /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf
nobody    5392  0.0  0.3  49036  3900 ?        S    15:25   0:00 nginx: worker p                                          rocess
nobody    5393  0.0  0.3  49036  3900 ?        S    15:25   0:00 nginx: worker p                                          rocess
root      5428  0.0  0.0 112704   976 pts/0    R+   15:25   0:00 grep --color=au                                          to nginx
[root@test-a ~]#

backup配置启动服务

[root@centos0 ~]# setenforce 0
[root@centos0 ~]# systemctl stop firewalld

[root@centos0 ~]# vim /usr/local/sbin/check_ng.sh
[root@centos0 ~]# cat /usr/local/sbin/check_ng.sh
#!/bin/bash
#时间变量，用于记录日志
d=`date --date today +%Y%m%d_%H:%M:%S`
#计算nginx进程数量
n=`ps -C nginx --no-heading|wc -l`
#如果进程为0，则启动nginx，并且再次检测nginx进程数量，
#如果还为0，说明nginx无法启动，此时需要关闭keepalived
if [ $n -eq "0" ]; then
        systemctl start nginx
        n2=`ps -C nginx --no-heading|wc -l`
        if [ $n2 -eq "0"  ]; then
                echo "$d nginx down,keepalived will stop" >> /var/log/check_ng.log
                systemctl stop keepalived
        fi
fi
[root@centos0 ~]# chmod 755 /usr/local/sbin/check_ng.sh
[root@centos0 ~]# vim /etc/keepalived/keepalived.conf
[root@centos0 ~]# cat /etc/keepalived/keepalived.conf
global_defs {
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}

vrrp_script chk_nginx {
    script "/usr/local/sbin/check_ng.sh"
    interval 3
}

vrrp_instance VI_1 {
    state BACKUP
    interface eno16777736
    virtual_router_id 1
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass test111
    }
    virtual_ipaddress {
        192.168.77.100
    }

    track_script {
        chk_nginx
    }

}

[root@centos0 ~]# ps aux | grep keep
root      4219  0.0  0.0 112656   988 pts/0    R+   15:46   0:00 grep --color=auto keep
[root@centos0 ~]# ps aux | grep nginx
root      4221  0.0  0.0 112656   988 pts/0    R+   15:46   0:00 grep --color=auto nginx

# 可以看到，keepalived 和 nginx 都没启动起来，这时如果启动keepalived，nginx也启动起来，说明配置成功
[root@centos0 ~]# systemctl start keepalived
[root@centos0 ~]# ps aux | grep keep
root      4230  0.0  0.1 119100  1376 ?        Ss   15:48   0:00 /usr/sbin/keepalived -D
root      4231  0.0  0.3 123268  3096 ?        S    15:48   0:00 /usr/sbin/keepalived -D
root      4232  0.0  0.2 123268  2608 ?        S    15:48   0:00 /usr/sbin/keepalived -D
root      4262  0.0  0.0 112656   988 pts/0    R+   15:48   0:00 grep --color=auto keep
[root@centos0 ~]# ps aux | grep nginx # nginx也启动起来啦
root      4256  0.0  0.2 125388  2112 ?        Ss   15:48   0:00 nginx: master process /usr/sbin/nginx
nginx     4257  0.0  0.3 125776  3132 ?        S    15:48   0:00 nginx: worker process
root      4282  0.0  0.0 112656   992 pts/0    R+   15:48   0:00 grep --color=auto nginx

浏览器访问测试

访问master:

访问backup:

访问绑定的vip:

停掉master上的keepalived，再进行访问测试，发现已经切到backup了

[root@test-a ~]# systemctl stop keepalived.service