day48-考试系统项目集群部署

1. ✅考试系统项目集群架构图

|------|-------------------------------|
| 负载均衡 | 说明 |
| 7层负载 | 通过nginx对http请求进行转发(uri,ua,类型) |
| 4层负载 | 对端口负载均衡(后端) |

2. 📝环境准备

|---------|-------------|------------------------|
| 角色 | 主机 | ip |
| 负载均衡 | lb01/lb02 | 172.16.1.5/172.16.1.6 |
| 前端web集群 | web01/web02 | 172.16.1.7/172.16.1.8 |
| 后端web集群 | web03/web04 | 172.16.1.9/172.16.1.10 |
| 数据库 | db02 | 172.16.1.52 |

3. 🔐部署数据库(db02)

3.1. 安装数据库

复制代码
1.解压,环境准备
mkdir -p /app/tools/   /app/data/3306/
tar xf mysql-8.0.28-linux-glibc2.12-x86_64.tar.xz  -C /app/tools/
ln -s /app/tools/mysql-8.0.28-linux-glibc2.12-x86_64/ /app/tools/mysql

2.安装依赖
yum install ncurses ncurses-devel libaio-devel openssl openssl-devel -y

#3.配置文件,用户
useradd -s /sbin/nologin  -M mysql 

4.设置配置文件
cat>/etc/my.cnf<<'EOF'
[mysqld]
user=mysql 
basedir=/app/tools/mysql/
datadir=/app/data/3306/
port=3306 
socket=/tmp/mysql.sock 

[client]
socket=/tmp/mysql.sock
EOF

3.2. 数据库基础配置

复制代码
5.修改配置和数据目录的所有者.
chown mysql.mysql /etc/my.cnf  
chown -R mysql.mysql /app/data/3306
chown -R root.root  /app/tools/mysql/

6.配置PATH环境变量
echo 'export PATH=/app/tools/mysql/bin:$PATH' >>/etc/profile
source /etc/profile
#7.检查 不提示命令找不到就是正常的.
mysql -V

3.3. 初始化数据库

复制代码
#8.初始化
mysqld --initialize-insecure --user=mysql \
--basedir=/app/tools/mysql/ --datadir=/app/data/3306/
echo $?

--initialize-insecure   以不安全方式初始化,root空密码,不加这个选项就是安全,创建root随机密码.

--user=mysql 用户
--basedir=/app/tools/mysql/ 安装目录
--datadir=/app/data/3306/   指定数据目录
与配置文件中一致.

3.4. 设置开机自启动

bash 复制代码
9.拷贝已经准备好的启动管理文件
cp /app/tools/mysql/support-files/mysql.server  /etc/init.d/mysqld
chmod +x /etc/init.d/mysqld


10.修改启动脚本中的basedir和datadir
#basedir=
#datadir=

sed -i  '/^basedir=/s#basedir=#basedir=/app/tools/mysql/#g'  /etc/init.d/mysqld 
sed -i  '/^datadir=/s#datadir=#datadir=/app/data/3306/#g'  /etc/init.d/mysqld 

11.检查 
egrep '^basedir=|^datadir=' /etc/init.d/mysqld
basedir=/app/tools/mysql/
datadir=/app/data/3306/

12.开机自启动服务,运行服务 (有些提示可以忽略)
#/etc/init.d/mysqld  start  (备用)

systemctl enable mysqld
systemctl start mysqld

13.检查
ps -ef | grep mysql
ss -lntup | grep mysql

4. 🚀启动jar项目(web03/web04)

  • 准备application-prod.yml文件(正式环境)
  • 准备xzs-3.9.0.jar
bash 复制代码
1.先手动前台运行.
mkdir -p /app/code/exam/backend/
cd /app/code/exam/backend/ 

java -jar  xxx.jar 

2.前台运行
java -Duser.timezone=Asia/Shanghai -jar -Dspring.profiles.active=prod  xzs-3.9.0.jar
xxxx=prod  === 读取application-prod.yml配置文件
xxxx=dev   === 读取application-dev.yml配置文件

3.后台运行
nohup java -Duser.timezone=Asia/Shanghai -jar -Dspring.profiles.active=prod xzs-3.9.0.jar &

4.1. 🔄开机自启动(web03/web04)

  • 步骤
bash 复制代码
1.书写jar包管理脚本
bash /server/scripts/exam.sh start
运行nohup java -Duser.timezone=Asia/Shanghai -jar -Dspring.profiles.active=prod xzs-3.9.0.jar >/dev/null 2>&1 &

bash /server/scripts/exam.sh stop
关闭指定的服务

bash /server/scripts/exam.sh restart
先关闭然后再启动服务

bash /server/scripts/exam.sh status
检查是否运行

2.systemctl调用脚本
[Unit]
Description=exam
After=network.target 
[Service]
Type=forking
#EnvironmentFile=/etc/sysconfig/tomcat
ExecStart=bash /server/scripts/exam.sh start
ExecStop=bash /server/scripts/exam.sh stop
[Install]
WantedBy=multi-user.target
  • 脚本
bash 复制代码
#1.vars
service=xzs-3.9.0.jar
dir=/app/code/exam/backend/
choose=$1
logfile=${dir}exam.log
time=`date +%F_%T`
jar_file=${dir}${service}

#2.function
function srv_start() {
  pid=`ps -ef | grep -w xzs-3.9.0 | grep -v grep | awk '{print $2}'`
  pidcount=`ps -ef | grep -w xzs-3.9.0 | grep -v grep | awk '{print $2}' | wc -l`
  if [ -f ${jar_file} ];then
    if [ -z ${pid} ];then
      cd ${dir}
      nohup java -Duser.timezone=Asia/Shanghai -jar -Dspring.profiles.active=prod xzs-3.9.0.jar >/dev/null 2>&1 &
    fi
  else
    exit
  fi
  return $?
}

function srv_stop() {
  pid=`ps -ef | grep -w xzs-3.9.0 | grep -v grep | awk '{print $2}'`
  pidcount=`ps -ef | grep -w xzs-3.9.0 | grep -v grep | awk '{print $2}' | wc -l`
  if [ ${pidcount} -gt 0 ];then
    kill ${pid}
  fi
}

function srv_restart() {
  srv_stop
  sleep 1
  srv_start
}

function srv_status() {
  pid=`ps -ef | grep -w xzs-3.9.0 | grep -v grep | awk '{print $2}'`
  pidcount=`ps -ef | grep -w xzs-3.9.0 | grep -v grep | awk '{print $2}' | wc -l`
  if [ ${pidcount} -ge 1 ];then
    echo -e "\E[1;32mrunning pid:[${pid}] ${service}\E[0m"
  else
    echo -e "\E[1;31mstoped ${service}\E[0m"
  fi
}

#3.case
case "${choose}" in
  start) srv_start ;;
  stop) srv_stop ;;
  restart) srv_restart ;;
  status) srv_status ;;
  *) echo input error
esac

5. 🍀接入4层负载均衡(lb01/lb02)

5.1. 检查是否支持4层负载均衡

bash 复制代码
nginx -V |& grep stream
--with-stream

5.2. 负载均衡配置

bash 复制代码
1.修改yum源
vim /etc/yum.repos.d/nginx.repo 
[nginx-stable]
name=nginx stable repo
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=1
enabled=1
gpgkey=https://nginx.org/keys/nginx_signing.key
module_hotfixes=true

2.安装nginx
yum -y install nginx

3.修改nginx配置文件
vim /etc/nginx/nginx.conf
stream {
    upstream exam_pools {
        server 10.0.0.9:8000;
        server 10.0.0.10:8000;
        hash $remote_addr consistent;
    }
    log_format basic '$remote_addr [$time_local] '
                     '$protocol $status $bytes_sent $bytes_received '
                     '$session_time';

    access_log /var/log/nginx-l4.log basic;
    server {
        listen 8000;
        proxy_pass exam_pools;
    }
}

4.重启nginx
nginx -t
systemctl enable nginx --now
systemctl reload nginx

5.检查
ss -lntup | grep 8000

6. 🌏部署前端web(web01/web02)

bash 复制代码
1.修改yum源
vim /etc/yum.repos.d/nginx.repo 
[nginx-stable]
name=nginx stable repo
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=1
enabled=1
gpgkey=https://nginx.org/keys/nginx_signing.key
module_hotfixes=true

2.安装nginx
yum -y install nginx

3.修改配置文件
vim /etc/nginx/conf.d/exam.conf
server {
  listen 80;
  server_name admin.zhubl.xyz;
  root  /app/code/exam/front/admin/;
  location / {
    index index.html;
  }
  location /api/ {
    proxy_pass  http://10.0.0.4:8000;
  }
}

server {
  listen 80;
  server_name student.zhubl.xyz;
  root  /app/code/exam/front/student/;
  location / {
    index index.html;
  }
  location /api/ {
    proxy_pass  http://10.0.0.4:8000;
  }
}

4.创建目录
mkdir /app/code/exam/front/

5.拷贝前端代码到站点目录
unzip exam-web-前端.zip
mv exam-web-前端/* /app/code/exam/front/

6.重启nginx
nginx -t
systemctl enable nginx --now
systemctl reload nginx

7. 🍀接入7层负载(lb01/lb02)

bash 复制代码
1.修改yum源
vim /etc/yum.repos.d/nginx.repo 
[nginx-stable]
name=nginx stable repo
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=1
enabled=1
gpgkey=https://nginx.org/keys/nginx_signing.key
module_hotfixes=true

2.安装nginx
yum -y install nginx

3.修改nginx配置文件
vim /etc/nginx/conf.d/exam.conf
upstream l7_pools {
  server 10.0.0.7:80;
  server 10.0.0.8:80;
  hash $remote_addr consistent;
}
server {
  listen 80;
  server_name admin.zhubl.xyz;
  location / {
    proxy_pass http://l7_pools;
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Real-Ip $remote_addr;
  }
}
server {
  listen 80;
  server_name student.zhubl.xyz;
  location / {
    proxy_pass http://l7_pools;
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Real-Ip $remote_addr;
  }
}

4.重启nginx
nginx -t
systemctl enable nginx --now
systemctl reload nginx

8. 🚀接入高可用(lb01)

bash 复制代码
1.安装keepalived
yum -y install keepalived

2.备份配置文件
cp /etc/keepalived/keepalived.conf{,.bak}

3.修改配置文件
vim /etc/keepalived/keepalived.conf
global_defs {
   router_id lb01  
}
vrrp_script check_lb.sh {   
   script /server/scripts/check_lb.sh
   interval 2
   weight  1
   user root 
}

vrrp_instance vip_3 {    
    state MASTER    
    interface ens33
    virtual_router_id 51 
    priority 100    
    advert_int 1        
    authentication {   
        auth_type PASS
        auth_pass 1111  
    }
    virtual_ipaddress { 
        10.0.0.3 dev ens33 label ens33:1
    }
    track_script  {
      check_lb.sh 
    }
}

vrrp_instance vip_4 {
    state BACKUP
    interface ens33
    virtual_router_id 52
    priority 50
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        10.0.0.4 dev ens33 label ens33:2
    }
}

4.编写脚本
vim /server/scripts/check_lb.sh
#1.vars
count=`ps -ef | grep nginx | wc -l`

#2.stop keepalived
if [ ${count} -eq 1 ];then
	systemctl stop keepalived
fi

5.重启keepalived
systemctl restart keepalived.service

9. 🚀接入高可用(lb02)

bash 复制代码
1.安装keepalived
yum -y install keepalived

2.备份配置文件
cp /etc/keepalived/keepalived.conf{,.bak}

3.修改配置文件
vim /etc/keepalived/keepalived.conf
global_defs {
   router_id lb02
}
vrrp_script check_lb.sh {
   script /server/scripts/check_lb.sh
   interval 2
   weight  1
   user root 
}

vrrp_instance vip_3 {
    state BACKUP
    interface ens33
    virtual_router_id 51
    priority 50
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        10.0.0.3 dev ens33 label ens33:1
    }
}

vrrp_instance vip_4 {
    state MASTER
    interface ens33
    virtual_router_id 52
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        10.0.0.4 dev ens33 label ens33:2
    }
    track_script  {
      check_lb.sh
    }
}

4.编写脚本
vim /server/scripts/check_lb.sh
#1.vars
count=`ps -ef | grep nginx | wc -l`

#2.stop keepalived
if [ ${count} -eq 1 ];then
	systemctl stop keepalived
fi

5.重启keepalived
systemctl restart keepalived.service

10. 🌏浏览器访问

复制代码
http://student.zhubl.xyz/ 学生端 student 123456
http://admin.zhubl.xyz/   管理端 admin   123456