Nginx+Keepalive 实现高可用并启用健康检查模块

1. 目标效果

keepalived 负责监控 192.168.1.20 和 192.168.1.30 这两台负载均衡的服务器，并自动选择一台作为主服务器。用户访问 http://192.168.1.10 时，由主服务器接收该请求。当 keepalived 检测到主服务器不可访问时，会将备服务器升级为主服务器，从而实现高可用。

在主服务器中，通过 nginx（tengine）实现负载均衡，将访问请求分流到 192.168.1.100 和 192.168.1.200 这两台业务服务器。 nginx 中的健康检查模块会检测业务服务器状态，如果检测到 192.168.1.100 不可访问，则不再将访问请求发送给该服务器。

2. 部署 Keepalived

2.1 主机 IP

主机	IP
虚拟 IP	192.168.1.10
主服务器	192.168.1.20
备服务器	192.168.1.30

2.2 主服务器设置

官方配置说明文档：Keepalived for Linux

bash 复制代码

yun install vim epel-release keepalived nginx -y

cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak
cat > /etc/keepalived/keepalived.conf <<EOF
! Configuration File for keepalived

global_defs {
    router_id nginx01
}

vrrp_script check_nginx {
    script "/etc/keepalived/nginx_check.sh"
    interval 2   
    weight 2  
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass linalive
    }
    virtual_ipaddress {
        192.168.1.10
    }
    track_script {
        check_nginx
    }
}
EOF

touch /etc/keepalived/nginx_check.sh && chmod +x /etc/keepalived/nginx_check.sh
cat > /etc/keepalived/nginx_check.sh <<EOF
#!/bin/bash
if ! pgrep -x "nginx" > /dev/null; then
    systemctl restart nginx
    sleep 2
    if ! pgrep -x "nginx" > /dev/null; then
        pkill keepalived
    fi
fi
EOF

2.2 备服务器设置

bash 复制代码

yun install vim epel-release keepalived nginx -y

cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak
cat > /etc/keepalived/keepalived.conf <<EOF
! Configuration File for keepalived

global_defs {
    router_id nginx02
}

vrrp_script check_nginx {
    script "/etc/keepalived/nginx_check.sh"
    interval 2   
    weight 2  
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass linalive
    }
    virtual_ipaddress {
        192.168.1.10
    }
    track_script {
        check_nginx
    }
}
EOF

touch /etc/keepalived/nginx_check.sh && chmod +x /etc/keepalived/nginx_check.sh
cat > /etc/keepalived/nginx_check.sh <<EOF
#!/bin/bash
if ! pgrep -x "nginx" > /dev/null; then
    systemctl restart nginx
    sleep 2
    if ! pgrep -x "nginx" > /dev/null; then
        pkill keepalived
    fi
fi
EOF

3. 部署 Tengine （主备服务器）

3.1 准备 Tengine 压缩文件

下载 tengine 压缩文件，将文件上传到 /opt 文件夹下。下载地址：The Tengine Web Server

本文章编写时，最新版是：tengine-3.1.0.tar.gz

3.2 解压并编译

bash 复制代码

yum install -y gcc gcc-c++ make pcre-devel zlib-devel openssl-devel
tar zxvf /opt/tengine-3.1.0.tar.gz -C /opt
cd /opt/tengine-3.1.0

# configure 有众多的参数可设置，可使用 ./configure --help 进行查看
# 按照官方说法默认应该是开启了健康检查模块，但实测需要手动添加参数
./configure --add-module=modules/ngx_http_upstream_check_module/

make && make install

3.3 添加服务项

bash 复制代码

cat > /etc/systemd/system/nginx.service <<EOF
[Unit]
Description=The Tengine HTTP and reverse proxy server
After=network.target

[Service]
Type=forking
PIDFile=/usr/local/nginx/logs/nginx.pid
ExecStartPre=/usr/local/nginx/sbin/nginx -t
ExecStart=/usr/local/nginx/sbin/nginx
ExecReload=/usr/local/nginx/sbin/nginx -s reload
ExecStop=/usr/local/nginx/sbin/nginx -s stop
PrivateTmp=true

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload

3.4 编辑 nginx 配置文件

此处配置的内容项可参考官方文档：ngx_http_upstream_check_module

bash 复制代码

# tengine 默认的安装路径是 /usr/local/nginx
# 配置文件路径： /usr/local/nginx/conf/nginx.conf
# /favicon.ico 是接口地址，需替换成真实的 api 接口
worker_processes  auto;

events {
    worker_connections  1024;
}

http {
    upstream cluster1 {
        server 192.168.1.100:8082;
        server 192.168.1.200:8089;

        check interval=3000 rise=2 fall=5 timeout=1000 type=http;
        check_http_send "HEAD /favicon.ico HTTP/1.0\r\n\r\n";
        check_http_expect_alive http_2xx http_3xx;
    }


    server {
        listen 80;
        server_name localhost;
        location / {
            index Index.aspx;
            proxy_pass http://cluster1;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }


        location /status {
            check_status;
            access_log   off;
        }
    }
}

3.5 启动服务并访问

使用 systemctl start nginx启动服务，并访问 localhost:80/status查看健康检查报表页

4. 写在最后

参考文章：Nginx配置高可用（Nginx+keepalived）