1.1lvs简介
LVS:Linux Virtual Server,负载调度器,内核集成,章文嵩,阿里的四层SLB(Server LoadBalance)是基
于LVS+keepalived实现
LVS 官网: http://www.linuxvirtualserver.org/
LVS 相关术语
- VS: Virtual Server,负责调度
- RS:RealServer,负责真正提供服务
3.2 lvs****集群体系结构
工作原理:
VS根据请求报文的目标IP和目标协议及端口将其调度转发至某RS,根据调度算法来挑选RS
3.3.LVS****概念
VS:Virtual Server(调度器)
RS:Real Server (真实业务主机)
CIP:Client IP (客户端主机的ip)
VIP: Virtual serve IP VS外网的IP (对外开放的让客户访问的ip)
DIP: Director IP VS内网的IP (调度器负责访问内网的ip)
RIP: Real server IP (真实业务主机IP)
访问流程:CIP <--> VIP == DIP <--> RIP3.4.lvs****集群的类型
lvs-nat:
修改请求报文的目标IP,多目标IP的DNAT
lvs-dr: 操纵封装新的MAC地址
lvs-tun:
在原请求IP报文之外新加一个IP首部
lvs-fullnat: 修改请求报文的源和目标IP
3.4.1.nat****模式
Ivs-nat:
本质是多目标IP的DNAT,通过将请求报文中的目标地址和目标端口修改为某挑出的RS的RIP和
PORT实现转发
RIP和DIP应在同一个IP网络,且应使用私网地址;RS的网关要指向DIP
请求报文和响应报文都必须经由Director转发,Director易于成为系统瓶颈
支持端口映射,可修改请求报文的目标PORT
VS必须是Linux系统,RS可以是任意OS系统
3.4.2.DR模式
DR:Direct Routing,直接路由,LVS默认模式,应用最广泛,通过为请求报文重新封装一个MAC首部进行
转发,源MAC是DIP所在的接口的MAC,目标MAC是某挑选出的RS的RIP所在接口的MAC地址;源
IP/PORT,以及目标IP/PORT均保持不变
3.4.7.TUN模式(了解)
转发方式:不修改请求报文的IP首部(源IP为CIP,目标IP为VIP),而在原IP报文之外再封装一个IP首部
(源IP是DIP,目标IP是RIP),将报文发往挑选出的目标RS;RS直接响应给客户端(源IP是VIP,目标IP
是CIP)
4.1nat模式实验
4.1.1nat模式环境设定

注:我的nat模式网段为192.168.32.0 仅主机网段为192.168.88.0
VS主机中
bash
[root@vsnode ~]# vmset.sh eth0 192.168.32.100 vsnode
[root@vsnode ~]# vmset.sh eth1 192.168.88.100 vsnode noroute
RS1
bash
#设定网络
[root@RS1 ~]# vmset.sh eth0 192.168.88.10 RS1 noroute
[root@RS1 ~]# nmcli connection modify eth0 ipv4.gateway 192.168.88.100
[root@RS1 ~]# nmcli connection reload
[root@RS1 ~]# nmcli connection up eth0
[root@RS1 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.0.100 0.0.0.0 UG 100 0 0 eth0
192.168.0.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0
#设定访问业务真实数据
[root@RS1 ~]# dnf install httpd -y
[root@RS1 ~]# systemctl enable --now httpd
[root@RS1 ~]# echo RS1 - 192.168.0.10 > /var/www/html/index.html
RS2
bash
#设定网络
[root@RS1 ~]# vmset.sh eth0 192.168.88.20 RS1 noroute
[root@RS1 ~]# nmcli connection modify eth0 ipv4.gateway 192.168.88.100
[root@RS1 ~]# nmcli connection reload
[root@RS1 ~]# nmcli connection up eth0
[root@RS1 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.88.100 0.0.0.0 UG 100 0 0 eth0
192.168.0.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0
#设定访问业务真实数据
[root@RS1 ~]# dnf install httpd -y
[root@RS1 ~]# systemctl enable --now httpd
[root@RS1 ~]# echo RS2 - 192.168.0.20 > /var/www/html/index.html
在vs主机中测试环境
bash
[root@vsnode ~]# curl 192.168.88.10
RS1 - 192.168.0.10
[root@vsnode ~]# curl 192.168.88.20
RS2 - 192.168.0.20
运行结果图

NAT模式实现方法
vs
bash
#1,开启内核路由功能
[root@vsnode ~]# echo net.ipv4.ip_forward=1 >> /etc/sysctl.conf
[root@vsnode ~]# sysctl -p
net.ipv4.ip_forward = 1
#2.编写策略
[root@vsnode ~]# yum install ipvsadm -y
[root@vsnode ~]# ipvsadm -C
[root@vsnode ~]# ipvsadm -A -t 192.168.32.100:80 -s wrr
[root@vsnode ~]# ipvsadm -a -t 192.168.32.100:80 -r 192.168.0.10:80 -m -w 1
[root@vsnode ~]# ipvsadm -a -t 192.168.32.100:80 -r 192.168.0.20:80 -m -w 1
[root@vsnode ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.32.100:80 wrr
-> 192.168.0.10:80 Masq 1 0 0
-> 192.168.0.20:80 Masq 1 0 0
#测试
[root@vsnode ~]# for i in {1..10};do curl 192.168.32.100;done
RS2 - 192.168.0.20
RS1 - 192.168.0.10
RS2 - 192.168.0.20
RS1 - 192.168.0.10
RS2 - 192.168.0.20
RS1 - 192.168.0.10
RS2 - 192.168.0.20
RS1 - 192.168.0.10
RS2 - 192.168.0.20
RS1 - 192.168.0.10
#更改权重
[root@vsnode ~]# ipvsadm -e -t 192.168.32.100:80 -r 192.168.88.10:80 -m -w 2
[root@vsnode ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.32.100:80 wrr
-> 192.168.0.10:80 Masq 2 0 5
-> 192.168.0.20:80 Masq 1 0 5
#测试
[root@vsnode ~]# for i in {1..10};do curl 192.168.32.100;done
RS2 - 192.168.0.20
RS1 - 192.168.0.10
RS1 - 192.168.0.10
RS2 - 192.168.0.20
RS1 - 192.168.0.10
RS1 - 192.168.0.10
RS2 - 192.168.0.20
RS1 - 192.168.0.10
RS1 - 192.168.0.10
RS2 - 192.168.0.20
实验结果图

当其中一台RS断联后的效果图(RS2断联)

总结:可见当rs2断联后依旧能访问到rs1
4.2.DR模式实验过程
4.2.1环境流程图


4.2.1环境设定方式
bash
#在路由器中
[root@router ~]# systemctl disable --now ipvsadm.service
Removed "/etc/systemd/system/multi-user.target.wants/ipvsadm.service".
[root@router ~]# ipvsadm -C
#在路由器中
[root@router ~]# vmset.sh eth0 172.25.254.100 vsnode
[root@router ~]# vmset.sh eth1 192.168.0.100 vsnode noroute、
#设定内核路由功能
[root@router ~]# echo net.ipv4.ip_forward=1 >> /etc/sysctl.conf
[root@router ~]# sysctl -p
net.ipv4.ip_forward = 1
#数据转发策略
[root@router ~]# iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 192.168.88.100
[root@vsnode ~]# iptables -t nat -A POSTROUTING -o eth0 -j SNAT --to-source 192.168.32.100
bash
#vsnode 调度器
[root@vsnode ~]# vmset.sh eth0 192.168.88.50 vsnode norouter
[root@vsnode ~]# vim /etc/NetworkManager/system-connections/eth0.nmconnection
[connection]
id=eth0
type=ethernet
interface-name=eth0
[ipv4]
method=manual
address1==192.168.88.50/24,192.168.88.100
[root@vsnode ~]# cd /etc/NetworkManager/system-connections/
[root@vsnode system-connections]# cp -p eth0.nmconnection lo.nmconnection
[root@vsnode system-connections]# vim lo.nmconnection
[connection]
id=lo
type=loopback
interface-name=lo
[ipv4]
method=manual
address1==127.0.0.1/8
address2=192.168.88.200/32
[root@RS1 system-connections]# nmcli connection reload
[root@RS1 system-connections]# nmcli connection up eth0
连接已成功激活(D-Bus 活动路径:/org/freedesktop/NetworkManager/ActiveConnection/7)
[root@RS1 system-connections]# nmcli connection up lo
连接已成功激活(D-Bus 活动路径:/org/freedesktop/NetworkManager/ActiveConnection/8)
#检测
root@vsnode system-connections]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.88.100 0.0.0.0 UG 100 0 0 eth0
192.168.88.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0
192.168.88.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0
[root@vsnode system-connections]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 192.168.88.200/32 brd 192.168.0.255 scope global noprefixroute eth0
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:0c:29:41:e5:8b brd ff:ff:ff:ff:ff:ff
altname enp3s0
altname ens160
inet 192.168.88.50/24 brd 192.168.0.255 scope global secondary noprefixroute eth0
valid_lft forever preferred_lft forever
inet6 fe80::e40:8975:6b9:fea8/64 scope link noprefixroute
valid_lft forever preferred_lft forever
bash
#客户端
[root@client ~]# vmset.sh eth0 192.168.32.99 client norouter
连接已成功激活(D-Bus 活动路径:/org/freedesktop/NetworkManager/ActiveConnection/4)
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:0c:29:e5:75:af brd ff:ff:ff:ff:ff:ff
altname enp3s0
altname ens160
inet 172.25.254.99/24 brd 172.25.254.255 scope global noprefixroute eth0
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fee5:75af/64 scope link tentative noprefixroute
valid_lft forever preferred_lft forever
client
[root@norouter ~]# ipvsadm -A -t 192.168.88.200:80 -s rr
[root@norouter ~]# ipvsadm -a -t 192.168.88.200:80 -r 192.168.88.10 -g
[root@norouter ~]# ipvsadm -a -t 192.168.88.200:80 -r 192.168.88.20 -g
[root@norouter ~]# ipvsadm -ln
[root@client ~]# vim /etc/NetworkManager/system-connections/eth0.nmconnection
[connection]
id=eth0
type=ethernet
interface-name=eth0
[ipv4]
method=manual
address1=172.25.254.99/24,172.25.254.100
dns=8.8.8.8;
[root@client ~]# nmcli connection reload
[root@client ~]# nmcli connection up eth0
连接已成功激活(D-Bus 活动路径:/org/freedesktop/NetworkManager/ActiveConnection/5)
[root@client ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 196.168.32.100 0.0.0.0 UG 100 0 0 eth0
196.168.32.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0
#检测
[root@client ~]# ping 192.168.0.200
PING 192.168.0.200 (192.168.0.200) 56(84) 比特的数据。
64 比特,来自 192.168.0.200: icmp_seq=1 ttl=128 时间=1.08 毫秒
bash
#RS1
[root@RS1 ~]# vmset.sh eth0 192.168.0.10 RS1 noroute
[root@RS1 ~]# nmcli connection modify eth0 ipv4.gateway 192.168.0.100
[root@RS1 ~]# nmcli connection reload
[root@RS1 ~]# nmcli connection up eth0
[root@RS1 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.0.100 0.0.0.0 UG 100 0 0 eth0
192.168.0.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0
#在lo上设定vip
[root@RS1 ~]# cd /etc/NetworkManager/system-connections/
[root@RS1 system-connections]# cp -p eth0.nmconnection lo.nmconnection
[root@RS1 system-connections]# vim lo.nmconnection
[connection]
id=lo
type=loopback
interface-name=lo
[ethernet]
[ipv4]
address1=127.0.0.1/8
address2=192.168.0.200/32
method=manual
[root@RS1 system-connections]# nmcli connection reload
[root@RS1 system-connections]# nmcli connection up lo
连接已成功激活(D-Bus 活动路径:/org/freedesktop/NetworkManager/ActiveConnection/6)
[root@RS1 system-connections]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 192.168.0.200/32 scope global lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
#arp禁止响应
[root@rs1 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@rs1 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@rs1 ~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
[root@rs1 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
bash
#RS2
[root@RS2 ~]# vmset.sh eth0 192.168.0.20 RS2 noroute
[root@RS2 ~]# nmcli connection modify eth0 ipv4.gateway 192.168.0.100
[root@RS2 ~]# nmcli connection reload
[root@RS2 ~]# nmcli connection up eth0
[root@RS2 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.0.100 0.0.0.0 UG 100 0 0 eth0
192.168.0.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0
#在lo上设定vip
[root@RS2 ~]# cd /etc/NetworkManager/system-connections/
[root@RS2 system-connections]# cp -p eth0.nmconnection lo.nmconnection
[root@RS2 system-connections]# vim lo.nmconnection
[connection]
id=lo
type=loopback
interface-name=lo
[ethernet]
[ipv4]
address1=127.0.0.1/8
address2=192.168.0.200/32
method=manual
[root@RS2 system-connections]# nmcli connection reload
[root@RS2 system-connections]# nmcli connection up lo
连接已成功激活(D-Bus 活动路径:/org/freedesktop/NetworkManager/ActiveConnection/6)
[root@RS2 system-connections]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 192.168.0.200/32 scope global lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
#arp禁止响应
[root@rs2 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@rs2 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@rs2 ~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
[root@rs2 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce