目录
- 一、什么是集群
- 二、集群分类
- [三、LVS 的作用](#三、LVS 的作用)
- [四、LVS 的四种模式及原理](#四、LVS 的四种模式及原理)
- [4.1 NAT 模式](#4.1 NAT 模式)
- [4.2 DR 模式(直接路由)](#4.2 DR 模式(直接路由))
- [4.3 TUN 模式(IP 隧道)](#4.3 TUN 模式(IP 隧道))
- [4.4 FULLNAT 模式](#4.4 FULLNAT 模式)
- [五、LVS 的十三种调度算法](#五、LVS 的十三种调度算法)
- 六、多端口轮询问题
- 七、会话粘滞:持久连接方案
一、什么是集群
集群就是把多台独立的服务器组合在一起,对外提供一个统一的访问入口。客户端只需要访问这个入口,不用关心里面有多少台机器在工作。
#mermaid-svg-dfow7ZDTNCXorbsU{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-dfow7ZDTNCXorbsU .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-dfow7ZDTNCXorbsU .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-dfow7ZDTNCXorbsU .error-icon{fill:#552222;}#mermaid-svg-dfow7ZDTNCXorbsU .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-dfow7ZDTNCXorbsU .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-dfow7ZDTNCXorbsU .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-dfow7ZDTNCXorbsU .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-dfow7ZDTNCXorbsU .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-dfow7ZDTNCXorbsU .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-dfow7ZDTNCXorbsU .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-dfow7ZDTNCXorbsU .marker{fill:#333333;stroke:#333333;}#mermaid-svg-dfow7ZDTNCXorbsU .marker.cross{stroke:#333333;}#mermaid-svg-dfow7ZDTNCXorbsU svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-dfow7ZDTNCXorbsU p{margin:0;}#mermaid-svg-dfow7ZDTNCXorbsU .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-dfow7ZDTNCXorbsU .cluster-label text{fill:#333;}#mermaid-svg-dfow7ZDTNCXorbsU .cluster-label span{color:#333;}#mermaid-svg-dfow7ZDTNCXorbsU .cluster-label span p{background-color:transparent;}#mermaid-svg-dfow7ZDTNCXorbsU .label text,#mermaid-svg-dfow7ZDTNCXorbsU span{fill:#333;color:#333;}#mermaid-svg-dfow7ZDTNCXorbsU .node rect,#mermaid-svg-dfow7ZDTNCXorbsU .node circle,#mermaid-svg-dfow7ZDTNCXorbsU .node ellipse,#mermaid-svg-dfow7ZDTNCXorbsU .node polygon,#mermaid-svg-dfow7ZDTNCXorbsU .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-dfow7ZDTNCXorbsU .rough-node .label text,#mermaid-svg-dfow7ZDTNCXorbsU .node .label text,#mermaid-svg-dfow7ZDTNCXorbsU .image-shape .label,#mermaid-svg-dfow7ZDTNCXorbsU .icon-shape .label{text-anchor:middle;}#mermaid-svg-dfow7ZDTNCXorbsU .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-dfow7ZDTNCXorbsU .rough-node .label,#mermaid-svg-dfow7ZDTNCXorbsU .node .label,#mermaid-svg-dfow7ZDTNCXorbsU .image-shape .label,#mermaid-svg-dfow7ZDTNCXorbsU .icon-shape .label{text-align:center;}#mermaid-svg-dfow7ZDTNCXorbsU .node.clickable{cursor:pointer;}#mermaid-svg-dfow7ZDTNCXorbsU .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-dfow7ZDTNCXorbsU .arrowheadPath{fill:#333333;}#mermaid-svg-dfow7ZDTNCXorbsU .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-dfow7ZDTNCXorbsU .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-dfow7ZDTNCXorbsU .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-dfow7ZDTNCXorbsU .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-dfow7ZDTNCXorbsU .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-dfow7ZDTNCXorbsU .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-dfow7ZDTNCXorbsU .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-dfow7ZDTNCXorbsU .cluster text{fill:#333;}#mermaid-svg-dfow7ZDTNCXorbsU .cluster span{color:#333;}#mermaid-svg-dfow7ZDTNCXorbsU div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-dfow7ZDTNCXorbsU .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-dfow7ZDTNCXorbsU rect.text{fill:none;stroke-width:0;}#mermaid-svg-dfow7ZDTNCXorbsU .icon-shape,#mermaid-svg-dfow7ZDTNCXorbsU .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-dfow7ZDTNCXorbsU .icon-shape p,#mermaid-svg-dfow7ZDTNCXorbsU .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-dfow7ZDTNCXorbsU .icon-shape .label rect,#mermaid-svg-dfow7ZDTNCXorbsU .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-dfow7ZDTNCXorbsU .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-dfow7ZDTNCXorbsU .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-dfow7ZDTNCXorbsU :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 用户访问 VIP
LVS 调度器统一入口
RS1后端真实服务器
RS2后端真实服务器
二、集群分类
集群按用途分为三类:
| 类型 | 目的 | 常见技术 |
|---|---|---|
| 负载均衡集群 (LB Cluster) | 把请求分摊到多台机器,提升处理能力 | LVS Nginx HAProxy |
| 高可用集群 (HA Cluster) | 主节点宕机后备机自动接管 | Keepalived Heartbeat |
| 高性能集群 (HPC Cluster) | 多台机器并行计算 | MPI Hadoop |
LVS 属于负载均衡集群。
三、LVS 的作用
LVS(Linux Virtual Server)工作在 OSI 四层(传输层),基于 TCP/UDP 做请求转发,把客户端的请求按一定规则分发给后端的 RealServer。
核心角色
| 角色 | 全称 | 说明 |
|---|---|---|
| VS | Virtual Server | 调度器,运行 ipvsadm,接收客户端请求并转发 |
| RS | Real Server | 真实服务器,实际处理请求的节点 |
| VIP | Virtual IP | 对外暴露的虚拟 IP,客户端访问的地址 |
| DIP | Director IP | 调度器与 RS 通信的 IP |
| RIP | Real IP | RS 的真实 IP |
| CIP | Client IP | 客户端的 IP |
四、LVS 的四种模式及原理
4.1 NAT 模式
调度器充当 RS 的网关。请求进来时做 DNAT (VIP 转换为 RIP),响应回来时做 SNAT (RIP 转换为 VIP)。请求和响应都经过调度器。
#mermaid-svg-nrrZQFYcOCEpn0XO{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-nrrZQFYcOCEpn0XO .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-nrrZQFYcOCEpn0XO .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-nrrZQFYcOCEpn0XO .error-icon{fill:#552222;}#mermaid-svg-nrrZQFYcOCEpn0XO .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-nrrZQFYcOCEpn0XO .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-nrrZQFYcOCEpn0XO .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-nrrZQFYcOCEpn0XO .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-nrrZQFYcOCEpn0XO .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-nrrZQFYcOCEpn0XO .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-nrrZQFYcOCEpn0XO .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-nrrZQFYcOCEpn0XO .marker{fill:#333333;stroke:#333333;}#mermaid-svg-nrrZQFYcOCEpn0XO .marker.cross{stroke:#333333;}#mermaid-svg-nrrZQFYcOCEpn0XO svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-nrrZQFYcOCEpn0XO p{margin:0;}#mermaid-svg-nrrZQFYcOCEpn0XO .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-nrrZQFYcOCEpn0XO .cluster-label text{fill:#333;}#mermaid-svg-nrrZQFYcOCEpn0XO .cluster-label span{color:#333;}#mermaid-svg-nrrZQFYcOCEpn0XO .cluster-label span p{background-color:transparent;}#mermaid-svg-nrrZQFYcOCEpn0XO .label text,#mermaid-svg-nrrZQFYcOCEpn0XO span{fill:#333;color:#333;}#mermaid-svg-nrrZQFYcOCEpn0XO .node rect,#mermaid-svg-nrrZQFYcOCEpn0XO .node circle,#mermaid-svg-nrrZQFYcOCEpn0XO .node ellipse,#mermaid-svg-nrrZQFYcOCEpn0XO .node polygon,#mermaid-svg-nrrZQFYcOCEpn0XO .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-nrrZQFYcOCEpn0XO .rough-node .label text,#mermaid-svg-nrrZQFYcOCEpn0XO .node .label text,#mermaid-svg-nrrZQFYcOCEpn0XO .image-shape .label,#mermaid-svg-nrrZQFYcOCEpn0XO .icon-shape .label{text-anchor:middle;}#mermaid-svg-nrrZQFYcOCEpn0XO .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-nrrZQFYcOCEpn0XO .rough-node .label,#mermaid-svg-nrrZQFYcOCEpn0XO .node .label,#mermaid-svg-nrrZQFYcOCEpn0XO .image-shape .label,#mermaid-svg-nrrZQFYcOCEpn0XO .icon-shape .label{text-align:center;}#mermaid-svg-nrrZQFYcOCEpn0XO .node.clickable{cursor:pointer;}#mermaid-svg-nrrZQFYcOCEpn0XO .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-nrrZQFYcOCEpn0XO .arrowheadPath{fill:#333333;}#mermaid-svg-nrrZQFYcOCEpn0XO .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-nrrZQFYcOCEpn0XO .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-nrrZQFYcOCEpn0XO .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-nrrZQFYcOCEpn0XO .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-nrrZQFYcOCEpn0XO .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-nrrZQFYcOCEpn0XO .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-nrrZQFYcOCEpn0XO .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-nrrZQFYcOCEpn0XO .cluster text{fill:#333;}#mermaid-svg-nrrZQFYcOCEpn0XO .cluster span{color:#333;}#mermaid-svg-nrrZQFYcOCEpn0XO div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-nrrZQFYcOCEpn0XO .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-nrrZQFYcOCEpn0XO rect.text{fill:none;stroke-width:0;}#mermaid-svg-nrrZQFYcOCEpn0XO .icon-shape,#mermaid-svg-nrrZQFYcOCEpn0XO .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-nrrZQFYcOCEpn0XO .icon-shape p,#mermaid-svg-nrrZQFYcOCEpn0XO .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-nrrZQFYcOCEpn0XO .icon-shape .label rect,#mermaid-svg-nrrZQFYcOCEpn0XO .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-nrrZQFYcOCEpn0XO .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-nrrZQFYcOCEpn0XO .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-nrrZQFYcOCEpn0XO :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} ① 请求 CIP→VIP
② DNAT: CIP→RIP
② DNAT: CIP→RIP
③ 响应 RIP→CIP
③ 响应 RIP→CIP
④ SNAT: VIP→CIP
客户端
调度器 VSDNAT / SNAT
RS1GW 指向 VS
RS2GW 指向 VS
| 优点 | 缺点 |
|---|---|
| 配置简单 | 请求和响应都经过调度器 |
| 支持端口映射 (VIP:80 → RS:8080) | 调度器吞吐量受带宽限制 |
| RS 可以是任意操作系统 | 适合中小规模集群 |
实验验证
| 机器 | IP | 角色 |
|---|---|---|
| 机器A (vsnode) | 192.168.181.138 |
VS 调度器 |
| 机器B (LinkQian) | 192.168.181.137 |
RS1 + 别名 .139 作为 RS2 |
步骤 1 --- RS 端部署
bash
dnf install httpd -y
systemctl enable --now httpd
echo 'RS1 - 192.168.181.137' > /var/www/html/index.html
# 添加 RS2 IP 别名 & 网关指向 VS
ip addr add 192.168.181.139/24 dev ens160
ip route del default
ip route add default via 192.168.181.138 dev ens160
步骤 2 --- VS 端部署
bash
echo 'net.ipv4.ip_forward=1' >> /etc/sysctl.conf && sysctl -p
dnf install ipvsadm -y
ipvsadm -A -t 192.168.181.138:80 -s wrr
ipvsadm -a -t 192.168.181.138:80 -r 192.168.181.137:80 -m -w 1
ipvsadm -a -t 192.168.181.138:80 -r 192.168.181.139:80 -m -w 1
实际输出 (ipvsadm -Ln):
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.181.138:80 wrr
-> 192.168.181.137:80 Masq 1 0 0
-> 192.168.181.139:80 Masq 1 0 0
步骤 3 --- 权重调整
bash
ipvsadm -e -t 192.168.181.138:80 -r 192.168.181.137:80 -m -w 2
# RS1(权重2) 获取约 2/3 请求,RS2(权重1) 获取约 1/3
步骤 4 --- 规则持久化
bash
ipvsadm-save -n > /etc/sysconfig/ipvsadm
systemctl enable --now ipvsadm.service
4.2 DR 模式(直接路由)
调度器修改数据包的目标 MAC 地址 (不改 IP)转发给 RS,RS 处理完后直接将响应发给客户端。
关键技术点:
1. 为什么 RS 的 lo 上绑 VIP?
RS 收到目标 IP 为 VIP 的包时,若本地无此 IP 会丢弃。lo 绑 VIP/32 让 RS 认为自己持有该 IP,正常接收。
2. 为什么要 ARP 抑制?
防止 RS 响应 VIP 的 ARP 请求导致客户端绕过调度器直连 RS。
arp_ignore=1:只响应该接口收到的 ARParp_announce=2:应答只带最佳本地地址
#mermaid-svg-qAGygw7LezwdKpaF{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-qAGygw7LezwdKpaF .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-qAGygw7LezwdKpaF .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-qAGygw7LezwdKpaF .error-icon{fill:#552222;}#mermaid-svg-qAGygw7LezwdKpaF .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-qAGygw7LezwdKpaF .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-qAGygw7LezwdKpaF .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-qAGygw7LezwdKpaF .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-qAGygw7LezwdKpaF .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-qAGygw7LezwdKpaF .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-qAGygw7LezwdKpaF .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-qAGygw7LezwdKpaF .marker{fill:#333333;stroke:#333333;}#mermaid-svg-qAGygw7LezwdKpaF .marker.cross{stroke:#333333;}#mermaid-svg-qAGygw7LezwdKpaF svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-qAGygw7LezwdKpaF p{margin:0;}#mermaid-svg-qAGygw7LezwdKpaF .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-qAGygw7LezwdKpaF .cluster-label text{fill:#333;}#mermaid-svg-qAGygw7LezwdKpaF .cluster-label span{color:#333;}#mermaid-svg-qAGygw7LezwdKpaF .cluster-label span p{background-color:transparent;}#mermaid-svg-qAGygw7LezwdKpaF .label text,#mermaid-svg-qAGygw7LezwdKpaF span{fill:#333;color:#333;}#mermaid-svg-qAGygw7LezwdKpaF .node rect,#mermaid-svg-qAGygw7LezwdKpaF .node circle,#mermaid-svg-qAGygw7LezwdKpaF .node ellipse,#mermaid-svg-qAGygw7LezwdKpaF .node polygon,#mermaid-svg-qAGygw7LezwdKpaF .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-qAGygw7LezwdKpaF .rough-node .label text,#mermaid-svg-qAGygw7LezwdKpaF .node .label text,#mermaid-svg-qAGygw7LezwdKpaF .image-shape .label,#mermaid-svg-qAGygw7LezwdKpaF .icon-shape .label{text-anchor:middle;}#mermaid-svg-qAGygw7LezwdKpaF .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-qAGygw7LezwdKpaF .rough-node .label,#mermaid-svg-qAGygw7LezwdKpaF .node .label,#mermaid-svg-qAGygw7LezwdKpaF .image-shape .label,#mermaid-svg-qAGygw7LezwdKpaF .icon-shape .label{text-align:center;}#mermaid-svg-qAGygw7LezwdKpaF .node.clickable{cursor:pointer;}#mermaid-svg-qAGygw7LezwdKpaF .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-qAGygw7LezwdKpaF .arrowheadPath{fill:#333333;}#mermaid-svg-qAGygw7LezwdKpaF .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-qAGygw7LezwdKpaF .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-qAGygw7LezwdKpaF .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-qAGygw7LezwdKpaF .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-qAGygw7LezwdKpaF .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-qAGygw7LezwdKpaF .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-qAGygw7LezwdKpaF .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-qAGygw7LezwdKpaF .cluster text{fill:#333;}#mermaid-svg-qAGygw7LezwdKpaF .cluster span{color:#333;}#mermaid-svg-qAGygw7LezwdKpaF div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-qAGygw7LezwdKpaF .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-qAGygw7LezwdKpaF rect.text{fill:none;stroke-width:0;}#mermaid-svg-qAGygw7LezwdKpaF .icon-shape,#mermaid-svg-qAGygw7LezwdKpaF .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-qAGygw7LezwdKpaF .icon-shape p,#mermaid-svg-qAGygw7LezwdKpaF .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-qAGygw7LezwdKpaF .icon-shape .label rect,#mermaid-svg-qAGygw7LezwdKpaF .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-qAGygw7LezwdKpaF .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-qAGygw7LezwdKpaF .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-qAGygw7LezwdKpaF :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} ① 请求 CIP→VIP
② CIP→VIPMAC→RS1
② CIP→VIPMAC→RS2
③ VIP→CIP直回客户端
③ VIP→CIP直回客户端
客户端
调度器 VSlo: VIP/32修改 MAC, 不改 IP
RS1lo: VIP/32arp_ignore=1
RS2lo: VIP/32arp_ignore=1
| 优点 | 缺点 |
|---|---|
| 响应不经调度器,吞吐量大 | 调度器和 RS 必须在同一物理网段 |
| 适合大规模高并发场景 | 不支持端口映射 |
| RS 可以是任意操作系统 | 配置较复杂(VIP 共享 + ARP 抑制) |
实验验证
步骤 1 --- RS 端 DR 配置
bash
ip addr add 192.168.181.200/32 dev lo
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ARP 参数确认:
/proc/sys/net/ipv4/conf/all/arp_ignore = 1
/proc/sys/net/ipv4/conf/lo/arp_ignore = 1
步骤 2 --- VS 端 DR 配置
bash
ip addr add 192.168.181.200/32 dev lo
ipvsadm -A -t 192.168.181.200:80 -s rr
ipvsadm -a -t 192.168.181.200:80 -r 192.168.181.137:80 -g
ipvsadm -a -t 192.168.181.200:80 -r 192.168.181.139:80 -g
实际输出 (ipvsadm -Ln):
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.181.200:80 rr
-> 192.168.181.137:80 Route 1 0 0
-> 192.168.181.139:80 Route 1 0 0
步骤 3 --- 连接表验证
IPVS connection entries
pro expire state source virtual destination
TCP 00:59 SYN_RECV 192.168.181.200:60660 192.168.181.200:80 192.168.181.139:80
TCP 00:59 SYN_RECV 192.168.181.137:53588 192.168.181.138:80 192.168.181.137:80
SYN_RECV表示 LVS 正在转发。表中同时出现 RS1 (.137) 和 RS2 (.139),轮询调度正常工作。
4.3 TUN 模式(IP 隧道)
在 DR 基础上通过 IPIP 隧道封装解除同网段限制。调度器把原始包封装进外层 IP 包(目标 RS 的 RIP),RS 解封后处理,响应直回客户端。
#mermaid-svg-pTxqiSzfTAnwSyzQ{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-pTxqiSzfTAnwSyzQ .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-pTxqiSzfTAnwSyzQ .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-pTxqiSzfTAnwSyzQ .error-icon{fill:#552222;}#mermaid-svg-pTxqiSzfTAnwSyzQ .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-pTxqiSzfTAnwSyzQ .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-pTxqiSzfTAnwSyzQ .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-pTxqiSzfTAnwSyzQ .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-pTxqiSzfTAnwSyzQ .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-pTxqiSzfTAnwSyzQ .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-pTxqiSzfTAnwSyzQ .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-pTxqiSzfTAnwSyzQ .marker{fill:#333333;stroke:#333333;}#mermaid-svg-pTxqiSzfTAnwSyzQ .marker.cross{stroke:#333333;}#mermaid-svg-pTxqiSzfTAnwSyzQ svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-pTxqiSzfTAnwSyzQ p{margin:0;}#mermaid-svg-pTxqiSzfTAnwSyzQ .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-pTxqiSzfTAnwSyzQ .cluster-label text{fill:#333;}#mermaid-svg-pTxqiSzfTAnwSyzQ .cluster-label span{color:#333;}#mermaid-svg-pTxqiSzfTAnwSyzQ .cluster-label span p{background-color:transparent;}#mermaid-svg-pTxqiSzfTAnwSyzQ .label text,#mermaid-svg-pTxqiSzfTAnwSyzQ span{fill:#333;color:#333;}#mermaid-svg-pTxqiSzfTAnwSyzQ .node rect,#mermaid-svg-pTxqiSzfTAnwSyzQ .node circle,#mermaid-svg-pTxqiSzfTAnwSyzQ .node ellipse,#mermaid-svg-pTxqiSzfTAnwSyzQ .node polygon,#mermaid-svg-pTxqiSzfTAnwSyzQ .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-pTxqiSzfTAnwSyzQ .rough-node .label text,#mermaid-svg-pTxqiSzfTAnwSyzQ .node .label text,#mermaid-svg-pTxqiSzfTAnwSyzQ .image-shape .label,#mermaid-svg-pTxqiSzfTAnwSyzQ .icon-shape .label{text-anchor:middle;}#mermaid-svg-pTxqiSzfTAnwSyzQ .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-pTxqiSzfTAnwSyzQ .rough-node .label,#mermaid-svg-pTxqiSzfTAnwSyzQ .node .label,#mermaid-svg-pTxqiSzfTAnwSyzQ .image-shape .label,#mermaid-svg-pTxqiSzfTAnwSyzQ .icon-shape .label{text-align:center;}#mermaid-svg-pTxqiSzfTAnwSyzQ .node.clickable{cursor:pointer;}#mermaid-svg-pTxqiSzfTAnwSyzQ .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-pTxqiSzfTAnwSyzQ .arrowheadPath{fill:#333333;}#mermaid-svg-pTxqiSzfTAnwSyzQ .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-pTxqiSzfTAnwSyzQ .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-pTxqiSzfTAnwSyzQ .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-pTxqiSzfTAnwSyzQ .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-pTxqiSzfTAnwSyzQ .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-pTxqiSzfTAnwSyzQ .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-pTxqiSzfTAnwSyzQ .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-pTxqiSzfTAnwSyzQ .cluster text{fill:#333;}#mermaid-svg-pTxqiSzfTAnwSyzQ .cluster span{color:#333;}#mermaid-svg-pTxqiSzfTAnwSyzQ div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-pTxqiSzfTAnwSyzQ .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-pTxqiSzfTAnwSyzQ rect.text{fill:none;stroke-width:0;}#mermaid-svg-pTxqiSzfTAnwSyzQ .icon-shape,#mermaid-svg-pTxqiSzfTAnwSyzQ .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-pTxqiSzfTAnwSyzQ .icon-shape p,#mermaid-svg-pTxqiSzfTAnwSyzQ .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-pTxqiSzfTAnwSyzQ .icon-shape .label rect,#mermaid-svg-pTxqiSzfTAnwSyzQ .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-pTxqiSzfTAnwSyzQ .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-pTxqiSzfTAnwSyzQ .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-pTxqiSzfTAnwSyzQ :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} ① CIP→VIP
② 封装包 DIP→RIP
② 封装包 DIP→RIP
③ VIP→CIP 直回
③ VIP→CIP 直回
客户端
调度器 VSIPIP 隧道封装外层 DIP→RIP内层 CIP→VIP
RS1tunl0: VIP/32
RS2tunl0: VIP/32
| 优点 | 缺点 |
|---|---|
| 可跨网段,不受二层限制 | RS 需支持 IPIP 隧道 |
| 响应直达客户端 | 封装/解封装有性能开销 |
| MTU 需调整(隧道头占 20 字节) |
RS 端操作:
bash
modprobe ipip
ip tunnel add tunl0 mode ipip remote <调度器IP> local <RIP>
ip addr add VIP/32 dev tunl0
ip link set tunl0 up
调度器端用 -i 添加 RS:
bash
ipvsadm -a -t VIP:Port -r RIP:Port -i
4.4 FULLNAT 模式
同时改目标 IP 和源 IP:DNAT(VIP 转换为 RIP)+ SNAT(CIP 转换为 DIP)。RS 看到的请求都来自调度器,不需要改网关。
#mermaid-svg-lnKVTuB0Un9v65dt{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-lnKVTuB0Un9v65dt .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-lnKVTuB0Un9v65dt .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-lnKVTuB0Un9v65dt .error-icon{fill:#552222;}#mermaid-svg-lnKVTuB0Un9v65dt .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-lnKVTuB0Un9v65dt .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-lnKVTuB0Un9v65dt .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-lnKVTuB0Un9v65dt .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-lnKVTuB0Un9v65dt .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-lnKVTuB0Un9v65dt .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-lnKVTuB0Un9v65dt .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-lnKVTuB0Un9v65dt .marker{fill:#333333;stroke:#333333;}#mermaid-svg-lnKVTuB0Un9v65dt .marker.cross{stroke:#333333;}#mermaid-svg-lnKVTuB0Un9v65dt svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-lnKVTuB0Un9v65dt p{margin:0;}#mermaid-svg-lnKVTuB0Un9v65dt .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-lnKVTuB0Un9v65dt .cluster-label text{fill:#333;}#mermaid-svg-lnKVTuB0Un9v65dt .cluster-label span{color:#333;}#mermaid-svg-lnKVTuB0Un9v65dt .cluster-label span p{background-color:transparent;}#mermaid-svg-lnKVTuB0Un9v65dt .label text,#mermaid-svg-lnKVTuB0Un9v65dt span{fill:#333;color:#333;}#mermaid-svg-lnKVTuB0Un9v65dt .node rect,#mermaid-svg-lnKVTuB0Un9v65dt .node circle,#mermaid-svg-lnKVTuB0Un9v65dt .node ellipse,#mermaid-svg-lnKVTuB0Un9v65dt .node polygon,#mermaid-svg-lnKVTuB0Un9v65dt .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-lnKVTuB0Un9v65dt .rough-node .label text,#mermaid-svg-lnKVTuB0Un9v65dt .node .label text,#mermaid-svg-lnKVTuB0Un9v65dt .image-shape .label,#mermaid-svg-lnKVTuB0Un9v65dt .icon-shape .label{text-anchor:middle;}#mermaid-svg-lnKVTuB0Un9v65dt .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-lnKVTuB0Un9v65dt .rough-node .label,#mermaid-svg-lnKVTuB0Un9v65dt .node .label,#mermaid-svg-lnKVTuB0Un9v65dt .image-shape .label,#mermaid-svg-lnKVTuB0Un9v65dt .icon-shape .label{text-align:center;}#mermaid-svg-lnKVTuB0Un9v65dt .node.clickable{cursor:pointer;}#mermaid-svg-lnKVTuB0Un9v65dt .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-lnKVTuB0Un9v65dt .arrowheadPath{fill:#333333;}#mermaid-svg-lnKVTuB0Un9v65dt .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-lnKVTuB0Un9v65dt .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-lnKVTuB0Un9v65dt .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-lnKVTuB0Un9v65dt .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-lnKVTuB0Un9v65dt .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-lnKVTuB0Un9v65dt .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-lnKVTuB0Un9v65dt .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-lnKVTuB0Un9v65dt .cluster text{fill:#333;}#mermaid-svg-lnKVTuB0Un9v65dt .cluster span{color:#333;}#mermaid-svg-lnKVTuB0Un9v65dt div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-lnKVTuB0Un9v65dt .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-lnKVTuB0Un9v65dt rect.text{fill:none;stroke-width:0;}#mermaid-svg-lnKVTuB0Un9v65dt .icon-shape,#mermaid-svg-lnKVTuB0Un9v65dt .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-lnKVTuB0Un9v65dt .icon-shape p,#mermaid-svg-lnKVTuB0Un9v65dt .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-lnKVTuB0Un9v65dt .icon-shape .label rect,#mermaid-svg-lnKVTuB0Un9v65dt .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-lnKVTuB0Un9v65dt .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-lnKVTuB0Un9v65dt .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-lnKVTuB0Un9v65dt :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} ① CIP→VIP
② DIP→RIP
② DIP→RIP
③ RIP→DIP
③ RIP→DIP
④ DIP→CIP
客户端 (CIP)
调度器 VSDNAT: VIP→RIPSNAT: CIP→DIP
RS1GW: 任意
RS2GW: 任意
| 优点 | 缺点 |
|---|---|
| RS 无需特殊网络配置 | 请求和响应都经调度器 |
| 支持跨网段 | RS 看不到真实客户端 IP |
| 标准内核需打补丁 |
四种模式对比
| 模式 | 响应路径 | 跨网段 | 端口映射 | RS 网关要求 |
|---|---|---|---|---|
| NAT | 经调度器 | ✗ | ✓ | 指向调度器 DIP |
| DR | 直达客户端 | ✗ | ✗ | 正常配置 |
| TUN | 直达客户端 | ✓ | ✗ | 正常配置 |
| FULLNAT | 经调度器 | ✓ | ✓ | 无特殊要求 |
五、LVS 的十三种调度算法
静态算法(不考虑负载,4 种)
| 算法 | 全称 | 说明 |
|---|---|---|
rr |
Round Robin | 轮询,一人一次 |
wrr |
Weighted RR | 加权轮询,权重高的多分 |
sh |
Source Hashing | 按源 IP 哈希,同一客户端固定同一 RS |
dh |
Destination Hashing | 按目标 IP 哈希,同一目标固定同一 RS |
rr: A B C A B C A B C ...
wrr: A A B C A A B C ... (A 权重=2, B 权重=1, C 权重=1)
实验中的 wrr 实操:
bash
ipvsadm -A -t 192.168.181.138:80 -s wrr
ipvsadm -a -t 192.168.181.138:80 -r 192.168.181.137:80 -m -w 2
ipvsadm -a -t 192.168.181.138:80 -r 192.168.181.139:80 -m -w 1
动态算法(根据连接数/负载,7 种)
| 算法 | 全称 | 调度依据 |
|---|---|---|
lc |
Least-Connection | 选活动连接数最少的 RS |
wlc |
Weighted LC | lc 基础上融入权重,权重高的承担更多 |
lblc |
Locality-Based LC | 同目标 IP 优先同 RS,负载过高再溢出 |
lblcr |
LBLCR | lblc + 多 RS 可服务同一目标 |
sed |
Shortest Expected Delay | (ActiveConn + 1) / Weight,选比值最小 |
nq |
Never Queue | 有空闲 RS 直接分配,否则走 sed |
ovf |
Overflow-connection | RS 达连接上限时溢出给其他节点 |
关键公式:
lc: overhead = ActiveConn × 256 + InactiveConn
wlc: overhead = (ActiveConn × 256 + InactiveConn) / Weight
选型建议: 短连接用
rr/wrr,长连接用lc/wlc,需会话保持用sh或-p,缓存集群用dh/lblc。
六、多端口轮询问题
6.1 问题描述
Web 服务同时开 HTTP (80) 和 HTTPS (443),各自独立建 LVS 服务时:
TCP 192.168.181.200:80 rr → RS1, RS2
TCP 192.168.181.200:443 rr → RS1, RS2
两个独立服务的轮询计数器不同步,同一客户端先后访问 http 和 https 可能都落到同一台 RS。
6.2 解决方案
用 iptables mangle 表给 VIP:80 和 VIP:443 的数据包打上同一个数字标记,LVS 基于标记统一调度。
#mermaid-svg-TfGAhP3BFQd90fAg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-TfGAhP3BFQd90fAg .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-TfGAhP3BFQd90fAg .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-TfGAhP3BFQd90fAg .error-icon{fill:#552222;}#mermaid-svg-TfGAhP3BFQd90fAg .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-TfGAhP3BFQd90fAg .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-TfGAhP3BFQd90fAg .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-TfGAhP3BFQd90fAg .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-TfGAhP3BFQd90fAg .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-TfGAhP3BFQd90fAg .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-TfGAhP3BFQd90fAg .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-TfGAhP3BFQd90fAg .marker{fill:#333333;stroke:#333333;}#mermaid-svg-TfGAhP3BFQd90fAg .marker.cross{stroke:#333333;}#mermaid-svg-TfGAhP3BFQd90fAg svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-TfGAhP3BFQd90fAg p{margin:0;}#mermaid-svg-TfGAhP3BFQd90fAg .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-TfGAhP3BFQd90fAg .cluster-label text{fill:#333;}#mermaid-svg-TfGAhP3BFQd90fAg .cluster-label span{color:#333;}#mermaid-svg-TfGAhP3BFQd90fAg .cluster-label span p{background-color:transparent;}#mermaid-svg-TfGAhP3BFQd90fAg .label text,#mermaid-svg-TfGAhP3BFQd90fAg span{fill:#333;color:#333;}#mermaid-svg-TfGAhP3BFQd90fAg .node rect,#mermaid-svg-TfGAhP3BFQd90fAg .node circle,#mermaid-svg-TfGAhP3BFQd90fAg .node ellipse,#mermaid-svg-TfGAhP3BFQd90fAg .node polygon,#mermaid-svg-TfGAhP3BFQd90fAg .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-TfGAhP3BFQd90fAg .rough-node .label text,#mermaid-svg-TfGAhP3BFQd90fAg .node .label text,#mermaid-svg-TfGAhP3BFQd90fAg .image-shape .label,#mermaid-svg-TfGAhP3BFQd90fAg .icon-shape .label{text-anchor:middle;}#mermaid-svg-TfGAhP3BFQd90fAg .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-TfGAhP3BFQd90fAg .rough-node .label,#mermaid-svg-TfGAhP3BFQd90fAg .node .label,#mermaid-svg-TfGAhP3BFQd90fAg .image-shape .label,#mermaid-svg-TfGAhP3BFQd90fAg .icon-shape .label{text-align:center;}#mermaid-svg-TfGAhP3BFQd90fAg .node.clickable{cursor:pointer;}#mermaid-svg-TfGAhP3BFQd90fAg .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-TfGAhP3BFQd90fAg .arrowheadPath{fill:#333333;}#mermaid-svg-TfGAhP3BFQd90fAg .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-TfGAhP3BFQd90fAg .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-TfGAhP3BFQd90fAg .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-TfGAhP3BFQd90fAg .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-TfGAhP3BFQd90fAg .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-TfGAhP3BFQd90fAg .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-TfGAhP3BFQd90fAg .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-TfGAhP3BFQd90fAg .cluster text{fill:#333;}#mermaid-svg-TfGAhP3BFQd90fAg .cluster span{color:#333;}#mermaid-svg-TfGAhP3BFQd90fAg div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-TfGAhP3BFQd90fAg .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-TfGAhP3BFQd90fAg rect.text{fill:none;stroke-width:0;}#mermaid-svg-TfGAhP3BFQd90fAg .icon-shape,#mermaid-svg-TfGAhP3BFQd90fAg .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-TfGAhP3BFQd90fAg .icon-shape p,#mermaid-svg-TfGAhP3BFQd90fAg .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-TfGAhP3BFQd90fAg .icon-shape .label rect,#mermaid-svg-TfGAhP3BFQd90fAg .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-TfGAhP3BFQd90fAg .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-TfGAhP3BFQd90fAg .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-TfGAhP3BFQd90fAg :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} tcp/80 请求
iptables manglePREROUTING-m multiport --dports 80,443-j MARK --set-mark 6666
tcp/443 请求
ipvsadm -A -f 6666 -s rr基于标记统一调度
RS1 :0
RS2 :0
实验验证
bash
# 步骤 1:iptables 标记
iptables -t mangle -A PREROUTING \
-d 192.168.181.200 -p tcp \
-m multiport --dports 80,443 \
-j MARK --set-mark 6666
# 步骤 2:基于 FWM 创建 LVS 服务
ipvsadm -A -f 6666 -s rr
ipvsadm -a -f 6666 -r 192.168.181.137 -g
ipvsadm -a -f 6666 -r 192.168.181.139 -g
实际输出:
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
FWM 6666 rr
-> 192.168.181.137:0 Route 1 0 0
-> 192.168.181.139:0 Route 1 0 0
RS 端口显示
:0,表示匹配所有端口 ------ 标记 6666 已涵盖 80 和 443。
七、会话粘滞:持久连接方案
7.1 原理
LVS 的 -p 参数实现持久连接:指定时间内,同一 CIP 的所有请求(不限端口)都发给同一台 RS。
#mermaid-svg-HasqNMof95DhaG0H{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-HasqNMof95DhaG0H .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-HasqNMof95DhaG0H .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-HasqNMof95DhaG0H .error-icon{fill:#552222;}#mermaid-svg-HasqNMof95DhaG0H .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-HasqNMof95DhaG0H .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-HasqNMof95DhaG0H .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-HasqNMof95DhaG0H .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-HasqNMof95DhaG0H .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-HasqNMof95DhaG0H .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-HasqNMof95DhaG0H .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-HasqNMof95DhaG0H .marker{fill:#333333;stroke:#333333;}#mermaid-svg-HasqNMof95DhaG0H .marker.cross{stroke:#333333;}#mermaid-svg-HasqNMof95DhaG0H svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-HasqNMof95DhaG0H p{margin:0;}#mermaid-svg-HasqNMof95DhaG0H .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-HasqNMof95DhaG0H .cluster-label text{fill:#333;}#mermaid-svg-HasqNMof95DhaG0H .cluster-label span{color:#333;}#mermaid-svg-HasqNMof95DhaG0H .cluster-label span p{background-color:transparent;}#mermaid-svg-HasqNMof95DhaG0H .label text,#mermaid-svg-HasqNMof95DhaG0H span{fill:#333;color:#333;}#mermaid-svg-HasqNMof95DhaG0H .node rect,#mermaid-svg-HasqNMof95DhaG0H .node circle,#mermaid-svg-HasqNMof95DhaG0H .node ellipse,#mermaid-svg-HasqNMof95DhaG0H .node polygon,#mermaid-svg-HasqNMof95DhaG0H .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-HasqNMof95DhaG0H .rough-node .label text,#mermaid-svg-HasqNMof95DhaG0H .node .label text,#mermaid-svg-HasqNMof95DhaG0H .image-shape .label,#mermaid-svg-HasqNMof95DhaG0H .icon-shape .label{text-anchor:middle;}#mermaid-svg-HasqNMof95DhaG0H .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-HasqNMof95DhaG0H .rough-node .label,#mermaid-svg-HasqNMof95DhaG0H .node .label,#mermaid-svg-HasqNMof95DhaG0H .image-shape .label,#mermaid-svg-HasqNMof95DhaG0H .icon-shape .label{text-align:center;}#mermaid-svg-HasqNMof95DhaG0H .node.clickable{cursor:pointer;}#mermaid-svg-HasqNMof95DhaG0H .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-HasqNMof95DhaG0H .arrowheadPath{fill:#333333;}#mermaid-svg-HasqNMof95DhaG0H .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-HasqNMof95DhaG0H .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-HasqNMof95DhaG0H .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-HasqNMof95DhaG0H .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-HasqNMof95DhaG0H .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-HasqNMof95DhaG0H .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-HasqNMof95DhaG0H .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-HasqNMof95DhaG0H .cluster text{fill:#333;}#mermaid-svg-HasqNMof95DhaG0H .cluster span{color:#333;}#mermaid-svg-HasqNMof95DhaG0H div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-HasqNMof95DhaG0H .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-HasqNMof95DhaG0H rect.text{fill:none;stroke-width:0;}#mermaid-svg-HasqNMof95DhaG0H .icon-shape,#mermaid-svg-HasqNMof95DhaG0H .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-HasqNMof95DhaG0H .icon-shape p,#mermaid-svg-HasqNMof95DhaG0H .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-HasqNMof95DhaG0H .icon-shape .label rect,#mermaid-svg-HasqNMof95DhaG0H .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-HasqNMof95DhaG0H .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-HasqNMof95DhaG0H .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-HasqNMof95DhaG0H :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} tcp/80
tcp/443
tcp/3306
udp/53
Client A同一 CIP
持久连接模板(CIP, VIP, FWM) → RS1持久期内有效
RS1
内部维护
(CIP, VIP, FWM) → RS映射,超时时间由-p参数指定(单位秒)。
实验验证
bash
ipvsadm -A -f 6666 -s rr -p 1
ipvsadm -a -f 6666 -r 192.168.181.137 -g
ipvsadm -a -f 6666 -r 192.168.181.139 -g
实际输出:
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
FWM 6666 rr persistent 1
-> 192.168.181.137:0 Route 1 0 0
-> 192.168.181.139:0 Route 1 1 0
持久连接模板 (ipvsadm -Lnc):
IPVS connection entries
pro expire state source virtual destination
IP 00:58 NONE 192.168.181.200:0 0.0.26.10:0 192.168.181.139:0
| 字段 | 含义 |
|---|---|
IP 条目 |
持久连接模板,端口 0 覆盖所有端口 |
expire: 00:58 |
模板剩余 58 秒过期 |
NONE |
持久期内同一 CIP 所有请求 → 同一 RS |
7.2 防火墙标记 + 持久连接
bash
# 1. 防火墙标记:80+443 → 6666
iptables -t mangle -A PREROUTING -d 192.168.181.200 -p tcp \
-m multiport --dports 80,443 -j MARK --set-mark 6666
# 2. 基于 FWM 创建调度 + 持久连接
ipvsadm -A -f 6666 -s rr -p 1
ipvsadm -a -f 6666 -r 192.168.181.137 -g
ipvsadm -a -f 6666 -r 192.168.181.139 -g
多端口统一调度 →
-f防火墙标记,会话粘滞 →-p持久连接