lvs dr+keepalived

XMYX-02023-11-08 17:02

基于keepalived(主从+双主) + LVS(DR模型) + DNS实现http高可用集群

keepalived高可用主机IP：172.21.5.22和172.21.5.21

http服务高可用主机IP：172.21.5.16和172.21.5.18

VIP采用172.16.32.5

各虚拟机及主机名和IP对应关系如下所示：

虚拟机 主机名 IP地址
HA1 node1 172.21.5.22
HA2 node2 172.21.5.21
HA3 node3 172.21.5.16
HA4 node4 172.21.5.18

接下来我们先去配置好http高可用服务两台主机，然后再回来配置keepalived。既然是LVS的DR模型，那我们先来配置另外两台主机，来实现http高可用集群。

首先，打开另外两外两台提供http服务的虚拟机，这里是HA3和HA4。为了方便你也可以改为RS1和RS2。
在DR模型中，只添加个http服务还不行，还有许多需要修改，貌似不是很简单，好吧，为了节约时间，这里我们通过一个脚本实现修改各个数据。

bash 复制代码 
#!/bin/bash
#
# Script to start LVS DR real server.
# chkconfig: - 90 10
# description: LVS DR real server
#

. /etc/rc.d/init.d/functions

VIP=172.16.32.5 # 定义VIP
host=$(/bin/hostname)

case "$1" in
start)
    # Start LVS-DR real server on this machine.
    /sbin/ifconfig lo down
    /sbin/ifconfig lo up
    echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
    echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
    /sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up
    /sbin/route add -host $VIP dev lo:0
    ;;
stop)
    # Stop LVS-DR real server loopback device(s).
    /sbin/ifconfig lo:0 down
    echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
    echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
    ;;
status)
    # Status of LVS-DR real server.
    islothere=$(/sbin/ifconfig lo:0 | grep $VIP)
    isrothere=$(netstat -rn | grep "lo:0" | grep $VIP)
    if [ ! "$islothere" -o ! "$isrothere" ]; then
        # Either the route or the lo:0 device not found.
        echo "LVS-DR real server Stopped."
    else
        echo "LVS-DR real server Running."
    fi
    ;;
*)
    # Invalid entry.
    echo "$0: Usage: $0 {start|status|stop}"
    exit 1
    ;;
esac

记得该脚本需要在另一个主机上也要执行一下。

两台主机上都执行过上边的脚本后，验证下各参数是否已经修改：下面几个命令均在node3主机上执行，可在node3上使用ssh node4 'COMMAND'来验证下node4上各参数是否已修改

bash 复制代码 
[root@wh ~]# ifconfig    #在node3上执行该命令，查看是否有VIP，使用ssh node4 'ifconfig'命令查看node4上是否也有VIP
eth0 Link encap:Ethernet  HWaddr 00:0C:29:7F:8F:44
	inet addr:172.21.5.18Bcast:172.16.255.255Mask:255.255.0.0
	UP BROADCAST RUNNING MULTICAST  MTU:1500Metric:1
	RX packets:162748errors:0dropped:0overruns:0frame:0
	TX packets:2368errors:0dropped:0overruns:0carrier:0
	collisions:0txqueuelen:1000
	RX bytes:26502458(25.2MiB)  TX bytes:200681(195.9KiB)
	Interrupt:59Base address:0x2000
lo  Link encap:Local Loopback
	inet addr:127.0.0.1Mask:255.0.0.0
	UP LOOPBACK RUNNING  MTU:16436Metric:1
	RX packets:10errors:0dropped:0overruns:0frame:0
	TX packets:10errors:0dropped:0overruns:0carrier:0
	collisions:0txqueuelen:0
	RX bytes:666(666.0b)  TX bytes:666(666.0b)
	lo:0Link encap:Local Loopback
	inet addr:172.16.32.5Mask:255.255.255.255
	UP LOOPBACK RUNNING  MTU:16436Metric:1
[root@wh ~]# route -n   #查看是否有配置的VIP特定路由
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
172.16.32.50.0.0.0255.255.255.255UH    000lo
169.254.0.00.0.0.0255.255.0.0U     000eth0
172.16.0.00.0.0.0255.255.0.0U     000eth0
0.0.0.0172.16.0.10.0.0.0UG    000eth0
[root@wh ~]# cat /proc/sys/net/ipv4/conf/all/arp_ignore
1
[root@wh ~]# cat /proc/sys/net/ipv4/conf/all/arp_announce
2

到此，我们的两台RealServer都以配置完毕，而且其http服务也已正常工作。
现在去编辑我们的keepalived的配置文件，并修改成我们所需要的。

bash 复制代码 
#先在node1主机上进行修改：
[root@wh ~]# vim /etc/keepalived/keepalived.conf
global_defs {

}

vrrp_script chk_httpd {
    script "killall -0 httpd"
    interval 2
    weight -2
    fall 2
    rise 1
}

vrrp_script chk_schedown {
    script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0"
    interval 2
    weight -2
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 132
    priority 101
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass langdu
    }
    virtual_ipaddress {
        172.16.32.5/16 dev eth0 label eth0:0
    }
    track_script {
        chk_httpd
        chk_schedown
    }
    notify_master "/etc/keepalived/notify.sh master"
    notify_backup "/etc/keepalived/notify.sh backup"
    notify_fault "/etc/keepalived/notify.sh fault"
}

virtual_server 172.16.32.5 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    nat_mask 255.255.0.0
    protocol TCP

    real_server 172.21.5.16 80 {
        weight 1
        HTTP_GET {
            url {
                path /
                status_code 200
            }
            connect_timeout 2
            nb_get_retry 3
            delay_before_retry 2
        }
    }

    real_server 172.21.5.18 80 {
        weight 2
        HTTP_GET {
            url {
                path /
                status_code 200
            }
            connect_timeout 2
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}

下面这个脚本时实现健康检查用的。即上边用到的notify.sh脚本

bash 复制代码 
#!/bin/bash
# Author: onlyyou
# description: 一个通知脚本示例

ifalias=${2:-eth0:0}
interface=$(echo $ifalias | awk -F: '{print $1}')
vip=$(ip addr show $interface | grep $ifalias | awk '{print $2}')
contact='root@localhost'
workspace=$(dirname $0)

notify() {
    subject="$vip 地址切换为 $1"
    body="$vip 地址切换为 $1 $(date '+%F %H:%M:%S')"
    echo $body | mail -s "$1 过渡" $contact # 实现发送邮件
}

case "$1" in
master)
    notify master
    exit 0
    ;;
backup)
    notify backup
    /etc/rc.d/init.d/httpd restart
    exit 0
    ;;
fault)
    notify fault
    exit 0
    ;;
*)
    echo "用法: $(basename $0) {master|backup|fault}"
    exit 1
    ;;
esac
bash 复制代码 
scp /etc/keepalived/keepalived.conf node2:/etc/keepalived/
# 将配置文件发给另一个keepalived主机，这里发给node2，假如你的主机不是node2，请做相应修改
# 在node2主机上，修改刚传过来的keepalived的配置文件。这里只需修改两项即可：
state BACKUP          #设置该node2主机为backup路由

priority 100 #设定node2主机的优先级为100，低于node1
# 在node2主机上，我们只需修改上述两项即可。

修改完成后保存退出
现在我们去安装ipvsadm软件包，然后再启动keepalived服务

bash 复制代码 
ssh node2 'yum -y install ipvsadm'   #通过node1主机在node2上安装ipvsadm软件包
yum -y install ipvsadm   #在node1上安装软件包
安装完毕后启动我们的keepalived服务。
service keepalived start    #启动node1上的keepalived服务
ssh node2 'service keepalived start'   #在node1上启动node2的keepalived服务

查看下ipvs规则

bash 复制代码 
ipvsadm -l -n   #使用该命令，查看是否有ipvs规则，显示如下：
IP Virtual Server version 1.2.1(size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  172.16.32.5:80rr
-> 172.21.5.18:80Route   100
-> 172.21.5.16:80Route   100
bash 复制代码 
ifconfig    #查看下node1上的IP配置情况
eth0      Link encap:Ethernet  HWaddr 00:0C:29:9F:2F:AF
	inet addr:172.21.5.22Bcast:172.16.255.255Mask:255.255.0.0
	UP BROADCAST RUNNING MULTICAST  MTU:1500Metric:1
	RX packets:290653errors:1dropped:0overruns:0frame:0
	TX packets:13874errors:0dropped:0overruns:0carrier:0
	collisions:0txqueuelen:1000
	RX bytes:46393162(44.2MiB)  TX bytes:2014631(1.9MiB)
	Interrupt:59Base address:0x2000
eth0:0Link encap:Ethernet  HWaddr 00:0C:29:9F:2F:AF
	inet addr:172.16.32.5Bcast:0.0.0.0Mask:255.255.0.0
	UP BROADCAST RUNNING MULTICAST  MTU:1500Metric:1
	Interrupt:59Base address:0x2000
lo        Link encap:Local Loopback
	inet addr:127.0.0.1Mask:255.0.0.0
	UP LOOPBACK RUNNING  MTU:16436Metric:1
	RX packets:10errors:0dropped:0overruns:0frame:0
	TX packets:10errors:0dropped:0overruns:0carrier:0
	collisions:0txqueuelen:0
	RX bytes:666(666.0b)  TX bytes:666(666.0b)

配置的VIP也已生效，我们的ipvs规则已经实现。现在在我们的物理机上访问下172.16.32.5，看一下，是否可以访问，显示什么信息吧。
到目前为止，貌似我们还没有实现IP地址漂移。我们只需在master路由主机上，在相应目录下创建一个down文件即可实现手动漂移IP地址。
在node1上

bash 复制代码 
cd /etc/keepalived/#进入该目录
touch down    #创建该文件，用来实现手动漂移IP地址
bash 复制代码 
tail /var/log/messages    #停几秒钟后，查看日志
May 1619:15:32node1 Keepalived_vrrp[2816]: VRRP_Instance(VI_1) Entering MASTER STATE
May 1619:15:32node1 Keepalived_vrrp[2816]: VRRP_Instance(VI_1) setting protocol VIPs.
May 1619:15:32node1 Keepalived_vrrp[2816]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for172.16.32.5
May 1619:15:32node1 Keepalived_healthcheckers[2815]: Netlink reflector reports IP 172.16.32.5added
May 1619:15:32node1 Keepalived_vrrp[2816]: Netlink reflector reports IP 172.16.32.5added
May 1619:15:32node1 Keepalived_vrrp[2816]: VRRP_Instance(VI_1) Received higher prio advert    #收到更高优先级的通告信息
May 1619:15:32node1 Keepalived_vrrp[2816]: VRRP_Instance(VI_1) Entering BACKUP STATE     #进入backup状态
May 1619:15:32node1 Keepalived_vrrp[2816]: VRRP_Instance(VI_1) removing protocol VIPs.     #转移VIP地址
May 1619:15:32node1 Keepalived_healthcheckers[2815]: Netlink reflector reports IP 172.16.32.5removed
May 1619:15:32node1 Keepalived_vrrp[2816]: Netlink reflector reports IP 172.16.32.5removed
bash 复制代码 
ifconfig   #使用该命令，查看下node1主机的VIP是否存在，可看到已转移到其他主机
eth0      Link encap:Ethernet  HWaddr 00:0C:29:9F:2F:AF
	inet addr:172.21.5.22Bcast:172.16.255.255Mask:255.255.0.0
	UP BROADCAST RUNNING MULTICAST  MTU:1500Metric:1
	RX packets:347881errors:1dropped:0overruns:0frame:0
	TX packets:21333errors:0dropped:0overruns:0carrier:0
	collisions:0txqueuelen:1000
	RX bytes:50760780(48.4MiB)  TX bytes:2653767(2.5MiB)
	Interrupt:59Base address:0x2000
lo        Link encap:Local Loopback
	inet addr:127.0.0.1Mask:255.0.0.0
	UP LOOPBACK RUNNING  MTU:16436Metric:1
	RX packets:10errors:0dropped:0overruns:0frame:0
	TX packets:10errors:0dropped:0overruns:0carrier:0
	collisions:0txqueuelen:0
	RX bytes:666(666.0b)  TX bytes:666(666.0b)

> 在node2主机上查看其日志。
```bash
tail /var/log/messages   #查看node2的日志信息
May 1619:15:32node1 Keepalived_vrrp[2464]: VRRP_Instance(VI_1) forcing a new MASTER election
May 1619:15:32node1 Keepalived_vrrp[2464]: VRRP_Instance(VI_1) forcing a new MASTER election
May 1619:15:33node1 Keepalived_vrrp[2464]: VRRP_Instance(VI_1) Transition to MASTER STATE
May 1619:15:34node1 Keepalived_vrrp[2464]: VRRP_Instance(VI_1) Entering MASTER STATE    #进入master状态
May 1619:15:34node1 Keepalived_vrrp[2464]: VRRP_Instance(VI_1) setting protocol VIPs.    #设定VIP
May 1619:15:34node1 Keepalived_healthcheckers[2463]: Netlink reflector reports IP 172.16.32.5added
May 1619:15:34node1 avahi-daemon[3375]: Registering new address record for172.16.32.5on eth0.
May 1619:15:34node1 Keepalived_vrrp[2464]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for172.16.32.5
May 1619:15:34node1 Keepalived_vrrp[2464]: Netlink reflector reports IP 172.16.32.5added
May 1619:15:39node1 Keepalived_vrrp[2464]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for172.16.32.5

在在我们的物理机访问下，看能否访问。依然在浏览器地址栏输入172.16.32.5，可以看到，访问正常。

现在，我们删掉/etc/keepalived/down这个文件，看能否实现IP漂移回来。

bash 复制代码 
rm -f /etc/keepalived/down      #删除node1主机上该文件
bash 复制代码 
tail /var/log/messages    #查看日志信息
May 1619:15:32node1 Keepalived_vrrp[2816]: Netlink reflector reports IP 172.16.32.5removed
May 1619:27:54node1 Keepalived_vrrp[2816]: VRRP_Script(chk_schedown) succeeded
May 1619:27:55node1 Keepalived_vrrp[2816]: VRRP_Instance(VI_1) forcing a new MASTER election
May 1619:27:55node1 Keepalived_vrrp[2816]: VRRP_Instance(VI_1) forcing a new MASTER election
May 1619:27:56node1 Keepalived_vrrp[2816]: VRRP_Instance(VI_1) Transition to MASTER STATE
May 1619:27:57node1 Keepalived_vrrp[2816]: VRRP_Instance(VI_1) Entering MASTER STATE    #进入master状态
May 1619:27:57node1 Keepalived_vrrp[2816]: VRRP_Instance(VI_1) setting protocol VIPs.
May 1619:27:57node1 Keepalived_vrrp[2816]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for172.16.32.5
May 1619:27:57node1 Keepalived_healthcheckers[2815]: Netlink reflector reports IP 172.16.32.5added
May 1619:27:57node1 Keepalived_vrrp[2816]: Netlink reflector reports IP 172.16.32.5added

至此，我们已成功实现了keepalived的相关功能。以上演示的仅仅是主从模式下地址漂移。那我们能否实现在双主模式下实现地址漂移呢？答案是肯定的。

相关推荐
Estar.Lee19 分钟前
时间操作[计算时间差]免费API接口教程
android·网络·后端·网络协议·tcp/ip
友友马1 小时前
『 Linux 』网络层 - IP协议(一)
linux·网络·tcp/ip
码老白2 小时前
【老白学 Java】Warshipv2.0（二）
java·网络
HackKong2 小时前
小白怎样入门网络安全？
网络·学习·安全·web安全·网络安全·黑客
vmlogin虚拟多登浏览器2 小时前
虚拟浏览器可以应对哪些浏览器安全威胁？
服务器·网络·安全·跨境电商·防关联
澜世2 小时前
2024小迪安全基础入门第三课
网络·笔记·安全·网络安全
ZZZCY20034 小时前
华为ENSP--IP编址及静态路由配置
网络·华为
EasyCVR5 小时前
私有化部署视频平台EasyCVR宇视设备视频平台如何构建视频联网平台及升级视频转码业务？
大数据·网络·音视频·h.265
hgdlip5 小时前
主IP地址与从IP地址：深入解析与应用探讨
网络·网络协议·tcp/ip
珹洺5 小时前
C语言数据结构——详细讲解 双链表
c语言·开发语言·网络·数据结构·c++·算法·leetcode
热门推荐
01(欧拉)openEuler系统添加网卡文件配置流程、(欧拉)openEuler系统手动配置ipv6地址流程、(欧拉)openEuler系统网络管理说明02PyTorch机器学习实现液态神经网络03【HarmonyOS】HUAWEI DevEco Studio 下载地址汇总04玄机平台应急响应—webshell查杀05Coze扣子平台完整体验和实践（附国内和国际版对比）06Ubuntu 20.04使用Livox mid 360 测试 FAST_LIO07怎样让音频速度变慢？请跟随以下方法进行操作08Unity中PICO实现 隔空取物 和 接触抓取物体09RAG 实践- Ollama+RagFlow 部署本地知识库10【目标跟踪】相机运动补偿