LVS+Keepalived+HAProxy双主高可用负载均衡集群

实验说明

LVS+Keeplived(接受四层流量) -> HAProxy(进行七层处理) -> Nginx(web业务或者反向代理应用业务)
NFS可作为静态资源的挂载与实现后端业务的统一，可使用DRBD实现NFS的高可用，后续可添加Redis与Mysql来存储更复杂数据或实现会话保持
DNS轮询是为了实现一个大流量业务Keepalived双主热备下保证数据的均匀分发，依托双主VIP架构打破传统单主备机闲置弊端，两台负载均衡节点同时在线承载流量，硬件资源利用率最大化

实验架构图

实验环境

主机名	IP地址	OS	所需软件
LB1	172.25.254.50	RockyLinux9.6	Keepalived，HAProxy，LVS
LB2	172.25.254.60	RockyLinux9.6	Keepalived，HAProxy，LVS
RS1	172.25.254.10	RockyLinux9.6	Nginx，nfs-utils
RS2	172.25.254.20	RockyLinux9.6	Nginx，nfs-utils
NFS-DNS	172.25.254.66	RockyLinux9.6	nfs-utils，bind
client	172.25.254.67	RockyLinux9.6

DNS服务部署

bash 复制代码

[root@nfs-dns ~]# dnf install bind -y

[root@nfs-dns ~]# vim /etc/named.conf
options {
        listen-on port 53 { any; };
...
        allow-query     { any; };		
        forwarders      { 8.8.8.8;}；
...
zone "web.com" IN {
        type master;
        file "named.web.com";
};

[root@nfs-dns ~]# cp -p /var/named/named.localhost /var/named/named.web.com
[root@nfs-dns ~]# cat /var/named/named.web.com
$TTL 1D
@       IN SOA  dns.web.com. admin.web.com. (
                                        0       ; serial
                                        1D      ; refresh
                                        1H      ; retry
                                        1W      ; expire
                                        3H )    ; minimum
        NS      dns.web.com.
dns     A       127.0.0.1
www     A       172.25.254.100
www     A       172.25.254.200

[root@nfs-dns ~]# systemctl enable --now named

#测试
[root@nfs-dns ~]# nslookup www.web.com
Server:         ::1
Address:        ::1#53

Name:   www.web.com
Address: 172.25.254.100
Name:   www.web.com
Address: 172.25.254.200

NFS服务部署

bash 复制代码

[root@nfs-dns ~]# dnf install nfs-utils -y
[root@nfs-dns ~]# mkdir -p /data/web 
[root@nfs-dns ~]# echo '/data/web *(rw,no_root_squash,no_all_squash,sync)' > /etc/exports

[root@nfs-dns ~]# systemctl enable --now nfs-server.service

#测试
[root@nfs-dns ~]# exportfs -rv
exporting *:/data/web

#web节点进行挂载
[root@RS1 ~]# dnf install nginx -y
[root@RS2 ~]# dnf install nginx -y

[root@RS1 ~]# dnf install nfs-utils -y
[root@RS2 ~]# dnf install nfs-utils -y

[root@RS1 ~]# mount 172.25.254.66:/data/web /usr/share/nginx/html/
[root@RS2 ~]# mount 172.25.254.66:/data/web /usr/share/nginx/html/

[root@RS1 ~]# echo "172.25.254.66:/data/web /usr/share/nginx/html/ nfs defaults 0 0" >> /etc/fstab
[root@RS2 ~]# echo "172.25.254.66:/data/web /usr/share/nginx/html/ nfs defaults 0 0" >> /etc/fstab

HAProxy部署

bash 复制代码

[root@LB1 ~]# dnf install haproxy -y
[root@LB2 ~]# dnf install haproxy -y

#LB1与LB2
[root@LB1 ~]# vim /etc/haproxy/haproxy.cfg
......
frontend http_front
        bind            *:80
        default_backend web_servers
        
backend web_servers
        balance roundrobin
        server web01 172.25.254.10:80 check inter 3000 fall 3 rise 2
        server web02 172.25.254.20:80 check inter 3000 fall 3 rise 2

[root@LB1 ~]# systemctl enable --now haproxy.service

LVS+Keepalived部署

安装软件

bash 复制代码

[root@LB1 ~]# dnf install keepalived ipvsadm -y
[root@LB2 ~]# dnf install keepalived ipvsadm -y

DR模式配置VIP

由于使用的是DR模式，所以要给RS配置VIP

bash 复制代码

[root@RS1 ~]# cd /etc/NetworkManager/system-connections/
[root@RS1 system-connections]# cp -p eth0.nmconnection lo.nmconnection
[root@RS1 system-connections]# cat > lo.nmconnection << EOF
[connection]
id=lo
type=loopback
interface-name=lo

[ipv4]
method=manual
address1=127.0.0.1/8
address2=172.25.254.100/32
address3=172.25.254.200/32
EOF

[root@RS1 system-connections]# nmcli connection reload
[root@RS1 system-connections]# nmcli connection up lo
[root@RS1 system-connections]# ip a			#查看VIP是否添加成功


[root@RS2 ~]# cd /etc/NetworkManager/system-connections/
[root@RS2 system-connections]# cp -p eth0.nmconnection lo.nmconnection
[root@RS2 system-connections]# cat > lo.nmconnection << EOF
[connection]
id=lo
type=loopback
interface-name=lo

[ipv4]
method=manual
address1=127.0.0.1/8
address2=172.25.254.100/32
address3=172.25.254.200/32
EOF

[root@RS2 system-connections]# nmcli connection reload
[root@RS2 system-connections]# nmcli connection up lo
[root@RS2 system-connections]# ip a

RS配置arp抑制

bash 复制代码

[root@RS1 system-connections]# cat >> /etc/sysctl.conf <<EOF
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
EOF
[root@RS1 system-connections]# sysctl -p

[root@LB2 system-connections]# cat >> /etc/sysctl.conf <<EOF
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
EOF
[root@LB2 system-connections]# sysctl -p

配置Keepalived与LVS

bash 复制代码

[root@LB1 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id LB1
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 150
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        172.25.254.100/24
    }
}

vrrp_instance VI_2 {
    state BACKUP
    interface eth0
    virtual_router_id 52
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        172.25.254.200/24
    }
}

virtual_server 172.25.254.100 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    protocol TCP

    real_server 127.0.0.1 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
}

virtual_server 172.25.254.200 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    protocol TCP

    real_server 127.0.0.1 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
}

systemctl enable --now keepalived

lb02（主 VIP2，备 VIP1）

只需要改：

VI_1：state BACKUP priority 100
VI_2：state MASTER priority 150
router_id LB2

测试

测试业务能否通过DNS轮询解析到域名访问

bash 复制代码

[root@client ~]# cat /etc/resolv.conf
# Generated by NetworkManager
nameserver 172.25.254.66

[root@client ~]# nslookup www.web.com
Server:         172.25.254.66
Address:        172.25.254.66#53

Name:   www.web.com
Address: 172.25.254.100
Name:   www.web.com
Address: 172.25.254.200

[root@nfs-dns ~]# echo "Welcome to Web Server" > /data/web/index.html
[root@client ~]# curl www.web.com
Welcome to Web Server

测试VIP是否漂移

bash 复制代码

[root@LB2 ~]# systemctl stop keepalived.service