云原生(IP 透传,HAProxy 四层负载均衡实验)

实验简介

实验一：IP 透传实验（七层 + 四层）

该实验围绕 HAProxy 实现IP 透传（让后端服务器获取真实客户端 IP）展开，分为七层（HTTP）和四层（TCP）两种场景：

1. 七层 IP 透传（HTTP 层）

• 基础环境：HAProxy 作为七层负载均衡，后端挂载两台 Apache Web 服务器（192.168.0.10/20），默认配置下后端 Apache 日志仅能记录 HAProxy 的 IP（192.168.0.100），无法获取真实客户端 IP（172.25.254.1）。
• 配置要点 ：
  • HAProxy 侧：在defaults段添加option forwardfor except 127.0.0.0/8，开启 HTTP 头透传（X-Forwarded-For）；
  • Apache 侧：修改日志格式（LogFormat），采集%{X-Forwarded-For}i字段；
• 验证效果：Apache 日志中成功显示真实客户端 IP（172.25.254.1）。

2. 四层 IP 透传（TCP 层）

• 环境调整 ：替换后端为 Nginx，关闭 Apache，Nginx 启用proxy_protocol（四层代理协议）；
• 问题与解决 ：
  • 仅开启 Nginx 的proxy_protocol会导致 502 错误，需将 HAProxy 切换为mode tcp（四层模式），并给后端服务器添加send-proxy参数；
  • Nginx 侧修改日志格式（log_format），采集$proxy_protocol_addr字段；
• 验证效果：Nginx 日志中成功记录真实客户端 IP，且 HAProxy 四层负载均衡正常轮询（RS1/RS2 交替响应）。

实验二：HAProxy 四层负载均衡（MariaDB 场景）

该实验聚焦 HAProxy四层（TCP）负载均衡在数据库场景的应用，核心验证负载分发与故障备份能力：

1. 基础四层负载

• 环境：两台 MariaDB 服务器（192.168.0.10/20，分别配置 server_id=10/20），创建远程登录用户并授权；
• HAProxy 配置 ：监听 6663 端口，mode tcp，轮询（roundrobin）分发请求到两台数据库；
• 验证 ：客户端连接 HAProxy 的 6663 端口，多次查询@@server_id，结果交替显示 10/20，证明轮询生效。

2. backup（备份）参数验证

• 配置调整 ：给 192.168.0.20 的数据库添加backup参数，使其成为备用节点；
• 验证逻辑 ：
  • 主节点（192.168.0.10）正常时，所有请求均分发到主节点；
  • 关闭主节点 MariaDB 服务后，HAProxy 检测到故障，自动切换到备用节点（192.168.0.20）；
  • 主节点恢复后，请求重新切回主节点。

核心总结

• 实验一：区分七层 / 四层 IP 透传的实现方式，七层依赖 HTTP 头（X-Forwarded-For），四层依赖 proxy_protocol 协议；
• 实验二：验证 HAProxy 四层负载的轮询策略，以及backup参数实现的故障自动切换能力。

IP透传

七层IP透传

#实验环境
[root@haproxy yxs]# vim /etc/haproxy/haproxy.cfg
listen webcluster
    bind        *:80
    balance     roundrobin
    server haha 192.168.0.10:80 check inter 3s fall 3 rise 5 weight 1
    server hehe 192.168.0.20:80 check inter 3s fall 3 rise 5 weight 1
[root@haproxy yxs]# systemctl restart haproxy.service

#测试环境
[root@test yxs]# for i in {1..5}
> do
> curl 172.25.254.100
> done
webserver1 - 192.168.0.10
webserver2 - 192.168.0.20
webserver1 - 192.168.0.10
webserver2 - 192.168.0.20
webserver1 - 192.168.0.10

#在rs主机中默认是未开启透传功能的
[root@webserver2 yxs]# cat /etc/httpd/logs/access_log
192.168.0.100 - - [08/Feb/2026:14:39:19 +0800] "GET / HTTP/1.1" 200 26 "-" "curl/7.76.1"
192.168.0.100 - - [08/Feb/2026:14:39:19 +0800] "GET / HTTP/1.1" 200 26 "-" "curl/7.76.1"

#开启ip透传的方式
[root@haproxy yxs]# vim /etc/haproxy/haproxy.cfg
defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull
    option http-server-close
    option forwardfor       except 127.0.0.0/8				#开启haproxy透传功能
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000
[root@haproxy yxs]# systemctl restart haproxy.service

#在rs中设定采集透传IP
[root@webserver2 yxs]# vim /etc/httpd/conf/httpd.conf
201     LogFormat "%h %l %u %t \"%r\" %>s %b \"%{X-Forwarded-For}i\" \"%{Referer}i\" \"%{User-Agent}i    \"" combined
[root@webserver2 yxs]# systemctl restart httpd

#测试
[root@webserver2 yxs]# cat /etc/httpd/logs/access_log
192.168.0.100 - - [08/Feb/2026:14:39:19 +0800] "GET / HTTP/1.1" 200 26 "-" "curl/7.76.1"
192.168.0.100 - - [08/Feb/2026:14:39:19 +0800] "GET / HTTP/1.1" 200 26 "-" "curl/7.76.1"

四层IP透传

#环境设置
#在RS中把apache停止
[root@webserver1 yxs]# systemctl disable --now httpd
Removed "/etc/systemd/system/multi-user.target.wants/httpd.service".
[root@webserver2 yxs]# systemctl disable --now httpd
Removed "/etc/systemd/system/multi-user.target.wants/httpd.service".

#部署nginx
[root@webserver1 yxs]# dnf install nginx -y
[root@webserver2 yxs]# dnf install nginx -y
[root@webserver1 yxs]# echo RS1 - 192.168.0.10 > /usr/share/nginx/html/index.html
[root@webserver1 yxs]# systemctl enable --now nginx
Created symlink /etc/systemd/system/multi-user.target.wants/nginx.service → /usr/lib/systemd/system/nginx.service.
[root@webserver2 yxs]# echo RS2 - 192.168.0.20 > /usr/share/nginx/html/index.html
[root@webserver2 yxs]# systemctl enable --now nginx
Created symlink /etc/systemd/system/multi-user.target.wants/nginx.service → /usr/lib/systemd/system/nginx.service.

#测试环境
[root@test yxs]# for i in {1..5}; do curl 172.25.254.100; done
RS1 - 192.168.0.10
RS2 - 192.168.0.20
RS1 - 192.168.0.10
RS2 - 192.168.0.20
RS1 - 192.168.0.10

#启用nginx的四层访问控制
[root@webserver1 yxs]# vim /etc/nginx/nginx.conf
server {
        listen       80 proxy_protocol;			#启用四层访问控制
        listen       [::]:80;
        server_name  _;
        root         /usr/share/nginx/html;

        # Load configuration files for the default server block.
        include /etc/nginx/default.d/*.conf;

        error_page 404 /404.html;
        location = /404.html {
        }
[root@webserver1 yxs]# systemctl restart nginx.service
[root@webserver2 yxs]# vim /etc/nginx/nginx.conf
 server {
        listen       80 proxy_protocol;			#启用四层访问控制
        listen       [::]:80;
        server_name  _;
        root         /usr/share/nginx/html;

        # Load configuration files for the default server block.
        include /etc/nginx/default.d/*.conf;

        error_page 404 /404.html;
        location = /404.html {
        }
[root@webserver2 yxs]# systemctl restart nginx.service

#测试
[root@test yxs]# for i in {1..5}; do curl 172.25.254.100; done
<html><body><h1>502 Bad Gateway</h1>
The server returned an invalid or incomplete response.
</body></html>
<html><body><h1>502 Bad Gateway</h1>
The server returned an invalid or incomplete response.
</body></html>
<html><body><h1>502 Bad Gateway</h1>
The server returned an invalid or incomplete response.
</body></html>
<html><body><h1>502 Bad Gateway</h1>
The server returned an invalid or incomplete response.
</body></html>
<html><body><h1>502 Bad Gateway</h1>
The server returned an invalid or incomplete response.
</body></html>

出现上述报错标识nginx只支持四层访问

#设定haproxy访问4层
[root@haproxy yxs]# vim /etc/haproxy/haproxy.cfg
listen webcluster
    bind        *:80
    mode        tcp				#四层访问
    balance     roundrobin
    server haha 192.168.0.10:80 send-proxy check inter 3s fall 3 rise 5 weight 1
    server hehe 192.168.0.20:80 send-proxy check inter 3s fall 3 rise 5 weight 1
[root@haproxy yxs]# systemctl restart haproxy.service

#测试四层访问
[root@test yxs]#  for i in {1..5}; do curl 172.25.254.100; done
RS1 - 192.168.0.10
RS2 - 192.168.0.20
RS1 - 192.168.0.10
RS2 - 192.168.0.20
RS1 - 192.168.0.10
[root@webserver1 yxs]# cat /var/log/nginx/access.log
192.168.0.100 - - [08/Feb/2026:16:55:21 +0800] "GET / HTTP/1.1" 200 19 "-" "curl/7.76.1" "172.25.254.110"
192.168.0.100 - - [08/Feb/2026:16:55:21 +0800] "GET / HTTP/1.1" 200 19 "-" "curl/7.76.1" "172.25.254.110"
192.168.0.100 - - [08/Feb/2026:16:55:21 +0800] "GET / HTTP/1.1" 200 19 "-" "curl/7.76.1" "172.25.254.110"
192.168.0.100 - - [08/Feb/2026:17:03:50 +0800] "GET / HTTP/1.1" 200 19 "-" "curl/7.76.1" "-"
192.168.0.100 - - [08/Feb/2026:17:03:50 +0800] "GET / HTTP/1.1" 200 19 "-" "curl/7.76.1" "-"
192.168.0.100 - - [08/Feb/2026:17:03:50 +0800] "GET / HTTP/1.1" 200 19 "-" "curl/7.76.1" "-"

Harproxy的四层负载

环境设定

#部署mariadb数据库
[root@webserver1 yxs]# dnf install mariadb-server mariadb  -y
[root@webserver2 yxs]# dnf install mariadb-server mariadb  -y
[root@webserver1 yxs]# vim /etc/my.cnf.d/mariadb-server.cnf
[mysqld]
server_id=10			#设定数据库所在主机的id标识，在20上设定id为20
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
log-error=/var/log/mariadb/mariadb.log
pid-file=/run/mariadb/mariadb.pid
[root@webserver2 yxs]# vim /etc/my.cnf.d/mariadb-server.cnf
[mysqld]
server_id=10			#设定数据库所在主机的id标识，在20上设定id为20
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
log-error=/var/log/mariadb/mariadb.log
pid-file=/run/mariadb/mariadb.pid
[root@webserver1 yxs]# systemctl start mariadb.service
[root@webserver2 yxs]# systemctl start mariadb.service

#建立远程登录用户并授权
[root@webserver1 yxs]# mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 3
Server version: 10.5.27-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> CREATE USER 'lee'@'%' identified by 'lee';
Query OK, 0 rows affected (0.000 sec)

MariaDB [(none)]> GRANT ALL ON *.* TO 'lee'@'%';
Query OK, 0 rows affected (0.000 sec)

MariaDB [(none)]> quit
Bye
[root@webserver2 yxs]# mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 3
Server version: 10.5.27-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> CREATE USER 'lee'@'%' identified by 'lee';
Query OK, 0 rows affected (0.000 sec)

MariaDB [(none)]> GRANT ALL ON *.* TO 'lee'@'%';
Query OK, 0 rows affected (0.000 sec)

MariaDB [(none)]> quit
Bye

#测试
[root@webserver1 yxs]# mysql -ulee -plee -h 192.168.0.10
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 4
Server version: 10.5.27-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> quit
Bye
[root@webserver2 yxs]# mysql -ulee -plee -h 192.168.0.20
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 4
Server version: 10.5.27-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> quit
Bye

四层负载操作

[root@haproxy yxs]# vim /etc/haproxy/haproxy.cfg
listen mariadbcluster
    bind        *:6663
    mode        tcp
    balance     roundrobin
    server haha 192.168.0.10:3306  check inter 3s fall 3 rise 5 weight 1
    server hehe 192.168.0.20:3306  check inter 3s fall 3 rise 5 weight 1
[root@haproxy yxs]# systemctl restart haproxy.service

#检测端口
[root@haproxy yxs]# netstat -antlupe  | grep haproxy
tcp        0      0 0.0.0.0:6663            0.0.0.0:*               LISTEN      0          82922      3507/haproxy        
udp        0      0 0.0.0.0:50413           0.0.0.0:*                           985        84003      3507/haproxy        

#测试
root@test yxs]# mysql -ulee -plee -h172.25.254.100 -P 6663
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 5
Server version: 10.5.27-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> SELECT @@server_id;
+-------------+
| @@server_id |
+-------------+
|          10 |
+-------------+
1 row in set (0.001 sec)

MariaDB [(none)]> SELECT @@server_id;
ERROR 2013 (HY000): Lost connection to MySQL server during query
MariaDB [(none)]> quit
Bye
[root@test yxs]# mysql -ulee -plee -h172.25.254.100 -P 6663
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 6
Server version: 10.5.27-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> SELECT @@server_id;
+-------------+
| @@server_id |
+-------------+
|          20 |
+-------------+
1 row in set (0.001 sec)

MariaDB [(none)]> quit
Bye

backup参数

[root@haproxy yxs]# vim /etc/haproxy/haproxy.cfg
listen mariadbcluster
    bind        *:6663
    mode        tcp
    balance     roundrobin
    server haha 192.168.0.10:3306  check inter 3s fall 3 rise 5 weight 1
    server hehe 192.168.0.20:3306  check inter 3s fall 3 rise 5 weight 1 backup
[root@haproxy yxs]# systemctl restart haproxy.service
[root@test yxs]# mysql -ulee -plee -h172.25.254.100 -P 6663
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 8
Server version: 10.5.27-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> SELECT @@server_id;
+-------------+
| @@server_id |
+-------------+
|          10 |
+-------------+
1 row in set (0.001 sec)

MariaDB [(none)]> quit
Bye
[root@test yxs]# mysql -ulee -plee -h172.25.254.100 -P 6663
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 9
Server version: 10.5.27-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> SELECT @@server_id;
+-------------+
| @@server_id |
+-------------+
|          10 |
+-------------+
1 row in set (0.001 sec)

#关闭10的mariadb并等待1分钟
[root@webserver1 yxs]# systemctl stop mariadb
[root@test yxs]# mysql -ulee -plee -h172.25.254.100 -P 6663
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 10
Server version: 10.5.27-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> SELECT @@server_id;
+-------------+
| @@server_id |
+-------------+
|          20 |
+-------------+
1 row in set (0.001 sec)

MariaDB [(none)]> quit
Bye

#还原故障主机等待片刻
[root@test yxs]# mysql -ulee -plee -h172.25.254.100 -P 6663
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 3
Server version: 10.5.27-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> SELECT @@server_id;
+-------------+
| @@server_id |
+-------------+
|          10 |
+-------------+
1 row in set (0.001 sec)

MariaDB [(none)]> quit
Bye