keepailved+nginx+nfs高可用

Keepalived+Nginx+NFS实现高可用

角色	IP地址	主机名	系统
Nginx主 + Keepalived主	192.168.228.131	master	Red Hat/CentOS 8
Nginx备 + Keepalived备	192.168.228.132	backup	Red Hat/CentOS 8
NFS服务器	192.168.228.135	nfs	Red Hat/CentOS 8
VIP (虚拟IP)	192.168.228.136	-	-

第一步：环境准备

在所有IP节点上

bash 复制代码

# 1. 配置主机名
# 在 131 节点：
hostnamectl hostname master

# 在 132 节点：
hostnamectl hostname backup

# 在 135 节点：
hostnamectl hostname nfs

# 2. 配置 hosts 文件（所有节点）
cat >> /etc/hosts << EOF
192.168.228.131 master
192.168.228.132 backup
192.168.228.135 nfs
EOF

# 3. 关闭 SELinux（临时）
setenforce 0

# 永久关闭
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

# 4. 配置防火墙
systemctl start firewalld
systemctl enable firewalld

# 5. 更新系统
dnf update -y

第二步：配置NFS服务器（192.168.228.135）

2.1安装nfs服务

bash 复制代码

# 安装 NFS 相关软件包

dnf install nfs-utils  -y

# 创建共享目录

mkdir -p /data/web
chmod 755 /data/web

# 创建测试文件
echo "$(hostname -I)" > /data/web_content/index.html

2.2配置nfs共享

bash 复制代码

# 编辑 exports 文件
cat > /etc/exports << EOF
/data/web 192.168.228.131(rw,sync,no_root_squash)
/data/web 192.168.228.132(rw,sync,no_root_squash)
EOF

2.3启动并配置nfs服务

bash 复制代码

# 启动服务
systemctl enable --now nfs-server

# 查看共享
exportfs -v
showmount -e localhost

# 配置防火墙
firewall-cmd --permanent --add-service=nfs
firewall-cmd --reload

第三步：配置nginx节点（131和132）

1.3 搭建nginx服务

1、分别在两台服务器中安装nginx服务。

bash 复制代码

# 在master中安装
[root@master ~]# dnf install nginx -y


# 在backup中安装
[root@backup ~]# dnf install nginx -y

2、修改欢迎页

bash 复制代码

[root@master ~]# echo $(hostname -I) > /usr/share/nginx/html/index.html


[root@backup ~]# echo $(hostname -I) > /usr/share/nginx/html/index.html

3、启动nginx服务

bash 复制代码

[root@master ~]# systemctl start nginx

[root@backup ~]# systemctl start nginx

4、测试服务

bash 复制代码

[root@master ~]# curl localhost
192.168.72.30


[root@backup ~]# curl localhost
192.168.72.32

第四步搭建keepalived

1、分别在两台服务器中安装keepalived软件

bash 复制代码

[root@master ~]# dnf install keepalived -y

[root@backup ~]# dnf install keepalived -y

2、配置keepalived

2.1 配置master

bash 复制代码

[root@master ~]# vim /etc/keepalived/keepalived.conf

文件的内容修改如下：

bash 复制代码

global_defs {
   router_id master
}

vrrp_instance VI_1 {
    state MASTER
    interface ens160
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.72.100
    }
}

2.2 配置backup

bash 复制代码

[root@backup ~]# vim /etc/keepalived/keepalived.conf

文件的内容修改如下：

bash 复制代码

global_defs {
   router_id backup
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens160
    virtual_router_id 51
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.72.100
    }
}

3、启动服务

bash 复制代码

[root@master ~]# systemctl start keepalived.service

[root@backup ~]# systemctl start keepalived.service

1.5 运行测试

bash 复制代码

[root@backup ~]# curl 192.168.72.100
192.168.72.30

第五步实现高可用

1、编写健康检查脚本

bash 复制代码

[root@master ~]# vim /etc/keepalived/check_nginx.sh

脚本的内容如下：

bash 复制代码

#!/bin/bash
count=`ps -C nginx --no-header | wc -l`
if [ ${count} -eq 0 ]; then
        systemctl start nginx
        sleep 2
        if [ `ps -C nginx --no-header | wc -l` -eq 0 ]; then
                systemctl stop keepalived
        fi
fi

2、给这个脚本赋予可执行权限

bash 复制代码

[root@master ~]# ll /etc/keepalived/check_nginx.sh 
-rw-r--r--. 1 root root 201 Jan 13 11:37 /etc/keepalived/check_nginx.sh
[root@master ~]# chmod +x /etc/keepalived/check_nginx.sh
[root@master ~]# ll /etc/keepalived/check_nginx.sh 
-rwxr-xr-x. 1 root root 201 Jan 13 11:37 /etc/keepalived/check_nginx.sh

3、将这个文件发送到backup节点

bash 复制代码

[root@master ~]# scp -p /etc/keepalived/check_nginx.sh root@192.168.72.32:/etc/keepalived/
The authenticity of host '192.168.72.32 (192.168.72.32)' can't be established.
ED25519 key fingerprint is SHA256:s1BvgtBs1UxSKS+5fVxpZGEOB76pE1/J2MAZnhNW6Wo.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.72.32' (ED25519) to the list of known hosts.
root@192.168.72.32's password: 
check_nginx.sh                                                                                     100%  201   319.9KB/s   00:00

4、最后在backup节点上进行验证

bash 复制代码

[root@backup ~]# ll /etc/keepalived/check_nginx.sh 
-rwxr-xr-x. 1 root root 201 Jan 13 11:37 /etc/keepalived/check_nginx.sh

5、将编写的脚本写入到keepalived的配置文件中（注意：两台服务器的配置文件都需要修改）

5.1 修改master配置文件

bash 复制代码

[root@master ~]# vim /etc/keepalived/keepalived.conf

文件的内容修改如下：

bash 复制代码

global_defs {
   router_id master
}
#以下是增加的内容
vrrp_script chk_nginx {
    script "/etc/keepalived/check_nginx.sh"
    interval 2
    timeout 2
    weight -20
    fall 3
    rise 2
}

vrrp_instance VI_1 {
    state MASTER
    interface ens160
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    #这是增加的内容
    track_script {
        chk_nginx
    }
    virtual_ipaddress {
        192.168.72.100
    }
}

5.2 修改backup节点的配置文件

bash 复制代码

[root@backup ~]# vim /etc/keepalived/keepalived.conf

文件内容修改如下：

bash 复制代码

global_defs {
   router_id backup
}

vrrp_script chk_nginx {
    script "/etc/keepalived/check_nginx.sh"
    interval 2
    timeout 2
    weight -20
    fall 3
    rise 2
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens160
    virtual_router_id 51
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    track_script {
        chk_nginx
    }
    virtual_ipaddress {
        192.168.72.100
    }
}

6、重启keepalived服务

bash 复制代码

[root@master ~]# systemctl restart keepalived.service


[root@backup ~]# systemctl restart keepalived.service

2.6 编写检测脚本

1、编写脚本

bash 复制代码

[root@master ~]# vim /etc/keepalived/check_nfs.sh

脚本内容如下：

bash 复制代码

#!/bin/bash
step=1
for (( i=0; i<60; i=(i+step) )); do
        systemctl status nfs-server &> /dev/null
        if [ $? -ne 0 ]; then
                systemctl restart nfs-server
                systemctl status nfs-server &> /dev/null
                if [ $? -ne 0 ]; then
                        sysytemctl stop keepalived
                fi
        fi
        sleep $step
done

2、设置可执行权限

bash 复制代码

[root@master ~]# chmod +x /etc/keepalived/check_nfs.sh
[root@master ~]# ll /etc/keepalived/check_nfs.sh
-rwxr-xr-x. 1 root root 275 Jan 13 16:56 /etc/keepalived/check_nfs.sh

3、将脚本添加到配置文件中

bash 复制代码

[root@master ~]# vim /etc/keepalived/keepalived.conf

在文件中内容如下：

bash 复制代码

global_defs {
   router_id nfs1
}

vrrp_script chk_nfs {
   script "/etc/keepalived/check_nfs.sh"
   interval 2
   fall 3
   rise 2
}

vrrp_instance VI_1 {
    state MASTER
    interface ens160
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    track_script {
        chk_nfs
    }
    virtual_ipaddress {
        192.168.72.110
    }
}

4、将配置文件有脚本分必到slave中

bash 复制代码

[root@master ~]# scp /etc/keepalived/* 192.168.72.36:/etc/keepalived/
The authenticity of host '192.168.72.36 (192.168.72.36)' can't be established.
ED25519 key fingerprint is SHA256:s1BvgtBs1UxSKS+5fVxpZGEOB76pE1/J2MAZnhNW6Wo.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.72.36' (ED25519) to the list of known hosts.
root@192.168.72.36's password: 
check_nfs.sh                                                                                                                                      100%  275   554.7KB/s   00:00    
keepalived.conf                                                                                                                                   100%  422   863.2KB/s   00:00

5、修改配置文件

bash 复制代码

[root@slave ~]# ll /etc/keepalived/check_nfs.sh
-rwxr-xr-x. 1 root root 275 Jan 13 17:00 /etc/keepalived/check_nfs.sh
[root@slave ~]# vim /etc/keepalived/keepalived.conf

文件内容如下：

bash 复制代码

global_defs {
   router_id nfs2
}

vrrp_script chk_nfs {
   script "/etc/keepalived/check_nfs.sh"
   interval 2
   fall 3
   rise 2
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens160
    virtual_router_id 51
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    track_script {
        chk_nfs
    }
    virtual_ipaddress {
        192.168.72.110
    }
}

2.7 验证结果

1、重启keepalived

bash 复制代码

[root@master ~]# systemctl restart keepalived.service

[root@slave ~]# systemctl restart keepalived.service

7、功能测试

bash 复制代码

[root@master ~]# curl 192.168.72.100
192.168.72.30
[root@master ~]# curl 192.168.72.100
192.168.72.30
[root@master ~]# curl 192.168.72.100
192.168.72.30
[root@master ~]# systemctl stop nginx
[root@master ~]# curl 192.168.72.100
192.168.72.30
[root@master ~]# curl 192.168.72.100
192.168.72.30
[root@master ~]# curl 192.168.72.100
192.168.72.30
[root@master ~]# curl 192.168.72.100
192.168.72.30
[root@master ~]# systemctl stop nginx
[root@master ~]# systemctl stop nginx
[root@master ~]# curl 192.168.72.100
curl: (7) Failed to connect to 192.168.72.100 port 80: Connection refused
[root@master ~]# curl 192.168.72.100
192.168.72.30
[root@master ~]# curl 192.168.72.100
192.168.72.30