麒麟操作系统 Minio集群安装

Minio 集群搭建

文章目录

[1. 环境前准备工作](#1. 环境前准备工作)
- [1.1 磁盘分区](#1.1 磁盘分区)
- [1.2 磁盘格式化](#1.2 磁盘格式化)
- [1.3 磁盘挂载](#1.3 磁盘挂载)
- [1.4 关闭防火墙](#1.4 关闭防火墙)
- [1.5 创建用户](#1.5 创建用户)
[2. 安装启动Minio](#2. 安装启动Minio)
- [2.1 下载Minio](#2.1 下载Minio)
- [2.2 配置文件编写](#2.2 配置文件编写)
- - [2.2.1 主机一](#2.2.1 主机一)
  - [2.2.2 主机二](#2.2.2 主机二)
  - [2.2.3 主机三](#2.2.3 主机三)
- [2.3 编写启动服务（rpm 安装可不操作）](#2.3 编写启动服务（rpm 安装可不操作）)
- [2.4 启动Minio](#2.4 启动Minio)
[3. 安装nginx](#3. 安装nginx)
- [3.1 安装前提(可选)](#3.1 安装前提(可选))
- [3.2 安装nginx](#3.2 安装nginx)
- [3.3 修改nginx配置文件](#3.3 修改nginx配置文件)
[4. Minio 负载](#4. Minio 负载)
- [4.1 编写nginx 配置文件](#4.1 编写nginx 配置文件)
- [4.2 启动nginx](#4.2 启动nginx)

环境介绍

操作系统	Kylin Linux Advanced Server V10 (Lance)	Kylin Linux Advanced Server V10 (Lance)	Kylin Linux Advanced Server V10 (Lance)
内核版本	Linux 4.19.90-52.22.v2207.ky10.aarch64	Linux 4.19.90-52.22.v2207.ky10.aarch64	Linux 4.19.90-52.22.v2207.ky10.aarch64
IP	192.168.31.82	192.168.31.83	192.168.31.84
Minio	20220802235916.0.0.aarch64	20220802235916.0.0.aarch64	20220802235916.0.0.aarch64

1. 环境前准备工作

三个节点同时挂载200G的硬盘集群模式必须挂载新磁盘，与系统共用磁盘会导致启动失败

三台节点各添加200G硬盘

1.1 磁盘分区

sh 复制代码

# fdisk -l 查看当前硬盘的分区情况
# 硬盘分区
fdisk /dev/vdb
输入n 创建新的分区
输入回车
默认分区输入1，回车，或者 指定存储大小 默认也行
输入w回车保存


# 查看所以块设备(硬盘驱动器、固态硬盘、USB 驱动器)
lsblk

1.2 磁盘格式化

sh 复制代码

mkfs -t ext4 /dev/vdb1

1.3 磁盘挂载

sh 复制代码

# 创建挂载目录
mkdir -p /data/minio

# 挂载
mount /dev/vdb1 /data/minio

# 查看挂载是否成功
df -h

#/dev/vdb1                196G   61M  186G    1% /data/minio

# 设置分区在系统重启后自动挂载
vi /etc/fstab

/dev/vdb1              /data/minio                  ext4    defaults        0 0

# 重新加载配置
mount -a

1.4 关闭防火墙

sh 复制代码

systemctl stop firewalld
systemctl disable firewalld

1.5 创建用户

sh 复制代码

groupadd -r minio-user
useradd -M -r -g minio-user minio-user
chown minio-user:minio-user /data/minio/

2. 安装启动Minio

2.1 下载Minio

sh 复制代码

mkdir -p /opt/software/minio && cd /opt/software/minio
# 下载rpm包
wget https://dl.min.io/server/minio/release/linux-arm64/archive/minio-20220802235916.0.0.aarch64.rpm 

# 查询本机是否存在minio rpm包
rpm -e | grep minio 

# 移除依赖
# rpm -e --nodeps minio-20220802235916.0.0.aarch64.rpm
# 安装rpm
rpm -ivh minio-20220802235916.0.0.aarch64.rpm

# 查看minio 是否安装成功
[root@192.168.31.82 minio]# minio -version
minio version RELEASE.2022-08-02T23-59-16Z (commit-id=76f950c6632fe67b17e9f0cc00905d7093ef837d)
Runtime: go1.18.5 linux/arm64
License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
Copyright: 2015-2022 MinIO, Inc.

2.2 配置文件编写

注意如果系统存在多个ip 请在MINIO_OPTS --address 和 --console-address 指定指定ip

不然后面账号密码即使正确也登陆不了！！！

2.2.1 主机一

sh 复制代码

vi /etc/default/minio
cat > /etc/default/minio <<EOF
# 文件磁盘的位置 因为我们是集群节点是82-84 挂载文件夹是/data/minio,生产环境不要使用默认密码
MINIO_VOLUMES="http://192.168.31.8{2...4}:9000/data/minio"

# minio-console的地址 就是web界面控制台
MINIO_OPTS="--address 192.168.31.82:9000  --console-address 192.168.31.82:9001"

# console的登陆账号
MINIO_ROOT_USER=yfkj
# # console的登陆密码
MINIO_ROOT_PASSWORD=yfkj_2024
EOF

2.2.2 主机二

sh 复制代码

cat > /etc/default/minio <<EOF
MINIO_VOLUMES="http://192.168.31.8{2...4}:9000/data/minio"
# minio-console的地址 就是web界面控制台
MINIO_OPTS="--address 192.168.31.83:9000  --console-address 192.168.31.83:9001"

# console的登陆账号
MINIO_ROOT_USER=yfkj
# # console的登陆密码
MINIO_ROOT_PASSWORD=yfkj_2024
EOF

2.2.3 主机三

sh 复制代码

cat > /etc/default/minio <<EOF
MINIO_VOLUMES="http://192.168.31.8{2...4}:9000/data/minio"

# minio-console的地址 就是web界面控制台
MINIO_OPTS="--address 192.168.31.84:9000  --console-address 192.168.31.84:9001"

# console的登陆账号
MINIO_ROOT_USER=yfkj
# # console的登陆密码
MINIO_ROOT_PASSWORD=yfkj_2024
EOF

MINIO_VOLUMES="http://192.168.31.82:9000/data/minio http://192.168.31.83:9000/data/minio http://192.168.31.84:9000/data/minio"

2.3 编写启动服务（rpm 安装可不操作）

sh 复制代码

vi /etc/systemd/system/minio.service

[Unit]
Description=MinIO
Documentation=https://min.io/docs/minio/linux/index.html
Wants=network-online.target
After=network-online.target
AssertFileIsExecutable=/usr/local/bin/minio

[Service]
WorkingDirectory=/usr/local

User=minio-user
Group=minio-user
ProtectProc=invisible

EnvironmentFile=-/etc/default/minio
ExecStartPre=/bin/bash -c "if [ -z \"${MINIO_VOLUMES}\" ]; then echo \"Variable MINIO_VOLUMES not set in /etc/default/minio\"; exit 1; fi"
ExecStart=/usr/local/bin/minio server $MINIO_OPTS $MINIO_VOLUMES

# MinIO RELEASE.2023-05-04T21-44-30Z adds support for Type=notify (https://www.freedesktop.org/software/systemd/man/systemd.service.html#Type=)
# This may improve systemctl setups where other services use `After=minio.server`
# Uncomment the line to enable the functionality
# Type=notify

# Let systemd restart this service always
Restart=always

# Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=65536

# Specifies the maximum number of threads this process can create
TasksMax=infinity

# Disable timeout logic and wait until process is stopped
TimeoutStopSec=infinity
SendSIGKILL=no

[Install]
WantedBy=multi-user.target

# Built for ${project.name}-${project.version} (${project.name})

2.4 启动Minio

sh 复制代码

# 刷新配置 配置文件修改记得执行
# systemctl daemon-reload
# 启动
systemctl start minio.service

# 查看状态
systemctl status minio.service

# 停止
systemctl stop minio.service

# 重新启动
systemctl restart minio.service

# 设置开机自启动
systemctl enable minio.service

# 查看minio 日志
journalctl -f -u minio.service


# 然后查看日志 发现有 ble to initialize backend: Storage resources are insufficient for the write  
# 请删除 /data/minio/.minio.sys 文件重新执行  systemctl restart minio.service

3. 安装nginx

3.1 安装前提(可选)

sh 复制代码

# 检查电脑是否安装了 pcre-config openssl 如果没有则安装
# 下载pcre(其他)
wget http://downloads.sourceforge.net/project/pcre/pcre/8.37/pcre-8.37.tar.gz

# 解压
tar -zxvf pcre-8.37.tar.gz

# 进入目录
cd pcre-8.37

# 编译安装
./configure
make install

# 检测是否安装成功
pcre-config --version

# 下载 openssl(其他)
https://github.com/openssl/openssl/releases/download/openssl-3.2.1/openssl-3.2.1.tar.gz

# 解压
tar -zxvf openssl-3.2.1.tar.gz

# 进入目录
cd openssl-3.2.1

# 编译安装
./config 

make install

# 安装zlib
yum -y install zlib zlib-devel libtool

3.2 安装nginx

sh 复制代码

# 官方地址
https://nginx.org/en/download.html

# 下载
mkdir -p /opt/software/nginx && cd /opt/software/nginx
wget https://nginx.org/download/nginx-1.26.2.tar.gz

# 解压
tar -zxvf nginx-1.26.2.tar.gz

# 进入到nginx目录编译安装
cd nginx-1.26.2

# 编译安装
./configure --prefix=/usr/local/nginx-1.26.2 \
--user=root \
--group=root \
--with-http_ssl_module \
--with-http_v2_module \
--with-http_realip_module \
--with-http_stub_status_module \
--with-http_gzip_static_module \
--with-pcre \
--with-stream \
--with-stream_ssl_module \
--with-stream_realip_module \
--with-mail \
--with-mail_ssl_module

# 安装
make && make install

# 创建软连接文件 方便后面升级
ln -s /usr/local/nginx-1.26.2 /usr/local/nginx

# 配置环境变量
vi /etc/profile.d/nginx.sh

#!/bin/bash

export NGINX_HOME=/usr/local/nginx
export PATH="${NGINX_HOME}/sbin:$PATH"

# 生效
source /etc/profile.d/nginx.sh

# 编写nginx服务
vi /usr/lib/systemd/system/nginx.service
[Unit]
Description=The nginx HTTP and reverse proxy server
After=network-online.target remote-fs.target nss-lookup.target
Wants=network-online.target

[Service]
Type=forking
PIDFile=/usr/local/nginx/logs/nginx.pid
ExecStartPre=/usr/bin/rm -f /usr/local/nginx/logs/nginx.pid
ExecStartPre=/usr/local/nginx/sbin/nginx -t
ExecStart=/usr/local/nginx/sbin/nginx
ExecReload=/usr/local/nginx/sbin/nginx -s reload
KillSignal=SIGQUIT
TimeoutStopSec=5
KillMode=process
PrivateTmp=true

[Install]
WantedBy=multi-user.target

# 增加可执行权限
chmod a+x  /usr/lib/systemd/system/nginx.service 

# 重新加载
systemctl daemon-reload

# 查看nginx版本
[root@192.168.31.82 ~]# nginx -v
nginx version: nginx/1.26.2

3.3 修改nginx配置文件

sh 复制代码

# 清空nginx.conf
echo > /usr/local/nginx/conf/nginx.conf
# 创建配置文件存放的目录
mkdir -p /usr/local/nginx/conf.d

# 导入配置文件信息
cat >> /usr/local/nginx/conf/nginx.conf << EOF

#user  nobody;
worker_processes  1;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    #gzip  on;
    include /usr/local/nginx/conf.d/*.conf;
}
EOF

# 检查配置文件是否错误
# nginx -t
[root@192.168.31.82 ~]# nginx -t
nginx: the configuration file /usr/local/nginx-1.26.2/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx-1.26.2/conf/nginx.conf test is successful

4. Minio 负载

4.1 编写nginx 配置文件

sh 复制代码

# 提前创建日志存储文件
touch /usr/local/nginx/logs/minio_server.log

# 写入配置信息
vi /usr/local/nginx/conf.d/minio_load.conf


# 日志格式
log_format minio_log '$remote_addr - $remote_user [$time_local] "$request" '
                    '$status $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "$http_x_forwarded_for"';
# 七层负载                    
upstream minio_server {
   	server 192.168.31.82:9000;
  	server 192.168.31.83:9000;
  	server 192.168.31.84:9000;
   	# 使用轮询方式请求
   	least_conn;
}

server {
    listen 19000 ;
	server_name 192.168.31.82;
	access_log /usr/local/nginx/logs/minio_server.log minio_log;
    location / {
    	client_body_buffer_size 10M;
        client_max_body_size 10G;
        proxy_buffers 1024 4k;
        proxy_pass http://minio_server;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
     }
}

# 检查配置文件是否存在错误
[root@192.168.31.82 ~]# nginx -t
nginx: the configuration file /usr/local/nginx-1.26.2/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx-1.26.2/conf/nginx.conf test is successful

4.2 启动nginx

sh 复制代码

systemctl start nginx

麒麟操作系统 Minio集群安装

Minio 集群搭建

文章目录

1. 环境前准备工作

1.1 磁盘分区

1.2 磁盘格式化

1.3 磁盘挂载

1.4 关闭防火墙

1.5 创建用户

2. 安装启动Minio

2.1 下载Minio

2.2 配置文件编写

2.2.1 主机一

2.2.2 主机二

2.2.3 主机三

2.3 编写启动服务 （rpm 安装可不操作）

2.4 启动Minio

3. 安装nginx

3.1 安装前提(可选)

3.2 安装nginx

3.3 修改nginx配置文件

4. Minio 负载

4.1 编写nginx 配置文件

4.2 启动nginx

2.3 编写启动服务（rpm 安装可不操作）