1 Minio介绍
MinIO 是一个基于 Apache License v2.0 开源协议的对象存储服务。它兼容亚马逊 S3 云存储服务接口,非常适合于存储大容量非结构化的数据,例如图片、视频、日志文件、备份数据和容器/虚拟机镜像等,而一个对象文件可以是任意大小,从几 kb 到最大 5T 不等。以下是 MinIO 的一些主要特点和优势:
-
兼容性: MinIO 使用标准的 Amazon S3 API,这意味着它与现有的 S3 应用程序和工具兼容,可以无缝替换 Amazon S3。
-
高性能: MinIO 借助于其分布式架构和优化的存储引擎,在处理大规模数据时表现出色。它可以水平扩展以满足各种工作负载需求。
-
轻量级: MinIO 的设计注重简单性和效率,因此它是一个轻量级的服务。这使得它易于部署、管理和维护。
-
高可用性: MinIO 支持数据的多副本复制和故障转移,确保数据的可靠性和高可用性。它可以在节点故障时自动进行数据修复和重平衡。
-
安全性: MinIO 提供多种安全功能,包括加密传输、访问控制列表 (ACLs)、策略管理和身份验证机制,以确保数据的保密性和完整性。
-
灵活性: MinIO 可以在各种环境中部署,包括本地数据中心、云环境和容器化环境。它支持多种存储后端,包括本地磁盘、分布式文件系统和对象存储。
2 Minio环境搭建
2.1 docker环境下
采用docker-compose搭建:
bash
# 可参考 https://docs.min.io/docs/minio-docker-quickstart-guide.html
version: '3'
services:
minio:
image: minio/minio:latest # 原镜像`minio/minio:latest`
container_name: minio # 容器名为'minio'
restart: unless-stopped # 指定容器退出后的重启策略为始终重启,但是不考虑在Docker守护进程启动时就已经停止了的容器
volumes: # 数据卷挂载路径设置,将本机目录映射到容器目录
- "./minio/data:/data"
- "./minio/minio:/minio"
- "./minio/config:/root/.minio"
environment: # 设置环境变量,相当于docker run命令中的-e
TZ: Asia/Shanghai
LANG: en_US.UTF-8
MINIO_PROMETHEUS_AUTH_TYPE: "public"
MINIO_ACCESS_KEY: "root" # 登录账号
MINIO_SECRET_KEY: "password" # 登录密码
command: server /data --console-address ":9001"
logging:
driver: "json-file"
options:
max-size: "100m"
ports: # 映射端口
- "9000:9000" # 文件上传&预览端口
- "9001:9001" # 控制台访问端口
启动服务:
bash
docker-compose -f docker-compose-minio.yml -p minio up -d
启动后:
访问地址:ip地址:9001/minio 登录账号密码:root/password
2.2 Linux环境下(非docker容器下)
下载并授权限:
bash
wget https://dl.min.io/sever/minio/release/linux-amd64/minio
chmod +x minio //添加执行权限
启动minio服务:
bash
MINIO_ROOT_USER=root MINIO_ROOT_PASSWORD=password ./minio server /data/minio/data --console-address ":9001"
- 用户名为"root"
- 密码为"password"
- 数据存储路径为"/data/minio/data"
- 控制台页面的访问端口为"9091"
2.3 Windows环境下
下载资源:
MinIO | S3 & Kubernetes Native Object Storage for AI
创建文件夹:
下载完成之后再合适的目录下创建三个必要的文件夹 ,分别是bin,data,logs文件夹
安装位置根据自身需求选择就好,我选在D盘的minio文件下
将minio.exe放入到bin中
然后执行命令(账号密码可通过命令行自行配置):
bash
D:\minio\bin\minio.exe server D:\minio\data --console-address ":9001" --address ":9000"
2.4 运行成功后访问
3 springboot整合代码实现
3.1 写代码前的准备
创建桶:
获取accessKey和secretKey:
3.2 编写代码
application.yaml:
bash
server:
port: 8088
minio:
endpoint: http://127.0.0.1:9000 #Minio服务所在地址
bucketName: miniotest #存储桶名称
accessKey: JW38e2AR9G5DgvmUCa51 #访问的key
secretKey: rK5zgSxAyUwqgIfSWmec9osxDPvyN2qoEqX3MxZa #访问的秘钥
pom.xml:
xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>springboot-demo</artifactId>
<groupId>com.wkf</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>minio</artifactId>
<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-autoconfigure</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.20</version>
</dependency>
<dependency>
<groupId>io.minio</groupId>
<artifactId>minio</artifactId>
<version>8.2.2</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.11</version>
</dependency>
</dependencies>
</project>
MinioConfig.java:
java
package com.wkf.minio.config;
import io.minio.MinioClient;
import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Data
@Configuration
@ConfigurationProperties(prefix = "minio")
public class MinioConfig {
private String endpoint;
private String accessKey;
private String secretKey;
private String bucketName;
@Bean
public MinioClient minioClient() {
return MinioClient.builder()
.endpoint(endpoint)
.credentials(accessKey, secretKey)
.build();
}
}
OSSController.java:
java
package com.wkf.minio.controller;
import com.wkf.minio.config.MinioConfig;
import com.wkf.minio.util.MinioUtils;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.apache.tomcat.util.http.fileupload.IOUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;
import org.springframework.web.multipart.MultipartFile;
import javax.servlet.http.HttpServletResponse;
import java.io.InputStream;
@Slf4j
@RestController
@RequestMapping("/oss")
public class OSSController {
@Autowired
private MinioUtils minioUtils;
@Autowired
private MinioConfig minioConfig;
/**
* file upload
*
* @param file
*/
@PostMapping("/upload")
public String upload(@RequestParam("file") MultipartFile file) {
try {
//file name
String fileName = file.getOriginalFilename();
String newFileName = System.currentTimeMillis() + "." + StringUtils.substringAfterLast(fileName, ".");
//type
String contentType = file.getContentType();
minioUtils.uploadFile(minioConfig.getBucketName(), file, newFileName, contentType);
return "upload success";
} catch (Exception e) {
e.printStackTrace();
log.error("upload fail");
return "upload fail";
}
}
/**
* delete
*
* @param fileName
*/
@DeleteMapping("/")
public void delete(@RequestParam("fileName") String fileName) {
minioUtils.removeFile(minioConfig.getBucketName(), fileName);
}
/**
* get file info
*
* @param fileName
* @return
*/
@GetMapping("/info")
public String getFileStatusInfo(@RequestParam("fileName") String fileName) {
return minioUtils.getFileStatusInfo(minioConfig.getBucketName(), fileName);
}
/**
* get file url
*
* @param fileName
* @return
*/
@GetMapping("/url")
public String getPresignedObjectUrl(@RequestParam("fileName") String fileName) {
return minioUtils.getPresignedObjectUrl(minioConfig.getBucketName(), fileName);
}
/**
* file download
*
* @param fileName
* @param response
*/
@GetMapping("/download")
public void download(@RequestParam("fileName") String fileName, HttpServletResponse response) {
try {
InputStream fileInputStream = minioUtils.getObject(minioConfig.getBucketName(), fileName);
response.setHeader("Content-Disposition", "attachment;filename=" + fileName);
response.setContentType("application/force-download");
response.setCharacterEncoding("UTF-8");
IOUtils.copy(fileInputStream, response.getOutputStream());
} catch (Exception e) {
log.error("download fail");
}
}
}
MinioUtil.java:
java
package com.wkf.minio.util;
import io.minio.*;
import io.minio.http.Method;
import io.minio.messages.Bucket;
import io.minio.messages.DeleteObject;
import io.minio.messages.Item;
import lombok.RequiredArgsConstructor;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.springframework.stereotype.Component;
import org.springframework.web.multipart.MultipartFile;
import sun.misc.BASE64Decoder;
import java.io.ByteArrayInputStream;
import java.io.InputStream;
import java.io.UnsupportedEncodingException;
import java.net.URLDecoder;
import java.util.*;
/**
* MinIO Utils
*
*/
@Slf4j
@Component
@RequiredArgsConstructor
public class MinioUtils {
private final MinioClient minioClient;
/****************************** Operate Bucket Start ******************************/
/**
* init Bucket when start SpringBoot container
* create bucket if the bucket is not exists
*
* @param bucketName
*/
@SneakyThrows(Exception.class)
private void createBucket(String bucketName) {
if (!bucketExists(bucketName)) {
minioClient.makeBucket(MakeBucketArgs.builder().bucket(bucketName).build());
}
}
/**
* verify Bucket is exist?,true:false
*
* @param bucketName
* @return
*/
@SneakyThrows(Exception.class)
public boolean bucketExists(String bucketName) {
return minioClient.bucketExists(BucketExistsArgs.builder().bucket(bucketName).build());
}
/**
* get Bucket strategy
*
* @param bucketName
* @return
*/
@SneakyThrows(Exception.class)
public String getBucketPolicy(String bucketName) {
return minioClient.getBucketPolicy(GetBucketPolicyArgs
.builder()
.bucket(bucketName)
.build());
}
/**
* get all Bucket list
*
* @return
*/
@SneakyThrows(Exception.class)
public List<Bucket> getAllBuckets() {
return minioClient.listBuckets();
}
/**
* Get related information based on bucketName
*
* @param bucketName
* @return
*/
@SneakyThrows(Exception.class)
public Optional<Bucket> getBucket(String bucketName) {
return getAllBuckets().stream().filter(b -> b.name().equals(bucketName)).findFirst();
}
/**
* Delete Bucket based on bucketName, true: deletion successful; false: deletion failed, file may no longer exist
*
* @param bucketName
* @throws Exception
*/
@SneakyThrows(Exception.class)
public void removeBucket(String bucketName) {
minioClient.removeBucket(RemoveBucketArgs.builder().bucket(bucketName).build());
}
/****************************** Operate Bucket End ******************************/
/****************************** Operate Files Start ******************************/
/**
* check file is exist
*
* @param bucketName
* @param objectName
* @return
*/
public boolean isObjectExist(String bucketName, String objectName) {
boolean exist = true;
try {
minioClient.statObject(StatObjectArgs.builder().bucket(bucketName).object(objectName).build());
} catch (Exception e) {
log.error("[MinioUtils]>>>>check file exist, Exception:", e);
exist = false;
}
return exist;
}
/**
* check directory exist?
*
* @param bucketName
* @param objectName
* @return
*/
public boolean isFolderExist(String bucketName, String objectName) {
boolean exist = false;
try {
Iterable<Result<Item>> results = minioClient.listObjects(
ListObjectsArgs.builder().bucket(bucketName).prefix(objectName).recursive(false).build());
for (Result<Item> result : results) {
Item item = result.get();
if (item.isDir() && objectName.equals(item.objectName())) {
exist = true;
}
}
} catch (Exception e) {
log.error("[MinioUtils]>>>>check file exist, Exception:", e);
exist = false;
}
return exist;
}
/**
* Query files based on file prefix
*
* @param bucketName
* @param prefix
* @param recursive
* @return MinioItem
*/
@SneakyThrows(Exception.class)
public List<Item> getAllObjectsByPrefix(String bucketName,
String prefix,
boolean recursive) {
List<Item> list = new ArrayList<>();
Iterable<Result<Item>> objectsIterator = minioClient.listObjects(
ListObjectsArgs.builder().bucket(bucketName).prefix(prefix).recursive(recursive).build());
if (objectsIterator != null) {
for (Result<Item> o : objectsIterator) {
Item item = o.get();
list.add(item);
}
}
return list;
}
/**
* get file InputStream
*
* @param bucketName
* @param objectName
* @return
*/
@SneakyThrows(Exception.class)
public InputStream getObject(String bucketName, String objectName) {
return minioClient.getObject(
GetObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.build());
}
/**
* Breakpoint download
*
* @param bucketName
* @param objectName
* @param offset
* @param length
* @return
*/
@SneakyThrows(Exception.class)
public InputStream getObject(String bucketName, String objectName, long offset, long length) {
return minioClient.getObject(
GetObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.offset(offset)
.length(length)
.build());
}
/**
* Get the list of files under the path
*
* @param bucketName
* @param prefix
* @param recursive
* @return
*/
public Iterable<Result<Item>> listObjects(String bucketName, String prefix, boolean recursive) {
return minioClient.listObjects(
ListObjectsArgs.builder()
.bucket(bucketName)
.prefix(prefix)
.recursive(recursive)
.build());
}
/**
* use MultipartFile to upload files
*
* @param bucketName
* @param file
* @param objectName
* @param contentType
* @return
*/
@SneakyThrows(Exception.class)
public ObjectWriteResponse uploadFile(String bucketName, MultipartFile file, String objectName, String contentType) {
InputStream inputStream = file.getInputStream();
return minioClient.putObject(
PutObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.contentType(contentType)
.stream(inputStream, inputStream.available(), -1)
.build());
}
/**
* picture upload
* @param bucketName
* @param imageBase64
* @param imageName
* @return
*/
public ObjectWriteResponse uploadImage(String bucketName, String imageBase64, String imageName) {
if (!StringUtils.isEmpty(imageBase64)) {
InputStream in = base64ToInputStream(imageBase64);
String newName = System.currentTimeMillis() + "_" + imageName + ".jpg";
String year = String.valueOf(new Date().getYear());
String month = String.valueOf(new Date().getMonth());
return uploadFile(bucketName, year + "/" + month + "/" + newName, in);
}
return null;
}
public static InputStream base64ToInputStream(String base64) {
ByteArrayInputStream stream = null;
try {
byte[] bytes = new BASE64Decoder().decodeBuffer(base64.trim());
stream = new ByteArrayInputStream(bytes);
} catch (Exception e) {
e.printStackTrace();
}
return stream;
}
/**
* upload local files
*
* @param bucketName
* @param objectName
* @param fileName
* @return
*/
@SneakyThrows(Exception.class)
public ObjectWriteResponse uploadFile(String bucketName, String objectName, String fileName) {
return minioClient.uploadObject(
UploadObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.filename(fileName)
.build());
}
/**
* upload files based on stream
*
* @param bucketName
* @param objectName
* @param inputStream
* @return
*/
@SneakyThrows(Exception.class)
public ObjectWriteResponse uploadFile(String bucketName, String objectName, InputStream inputStream) {
return minioClient.putObject(
PutObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.stream(inputStream, inputStream.available(), -1)
.build());
}
/**
* create file or direatory
*
* @param bucketName
* @param objectName
* @return
*/
@SneakyThrows(Exception.class)
public ObjectWriteResponse createDir(String bucketName, String objectName) {
return minioClient.putObject(
PutObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.stream(new ByteArrayInputStream(new byte[]{}), 0, -1)
.build());
}
/**
* get file info
*
* @param bucketName
* @param objectName
* @return
*/
@SneakyThrows(Exception.class)
public String getFileStatusInfo(String bucketName, String objectName) {
return minioClient.statObject(
StatObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.build()).toString();
}
/**
* copy file
*
* @param bucketName
* @param objectName
* @param srcBucketName
* @param srcObjectName
*/
@SneakyThrows(Exception.class)
public ObjectWriteResponse copyFile(String bucketName, String objectName, String srcBucketName, String srcObjectName) {
return minioClient.copyObject(
CopyObjectArgs.builder()
.source(CopySource.builder().bucket(bucketName).object(objectName).build())
.bucket(srcBucketName)
.object(srcObjectName)
.build());
}
/**
* delete file
*
* @param bucketName
* @param objectName
*/
@SneakyThrows(Exception.class)
public void removeFile(String bucketName, String objectName) {
minioClient.removeObject(
RemoveObjectArgs.builder()
.bucket(bucketName)
.object(objectName)
.build());
}
/**
* batch delete file
*
* @param bucketName
* @param keys
* @return
*/
public void removeFiles(String bucketName, List<String> keys) {
List<DeleteObject> objects = new LinkedList<>();
keys.forEach(s -> {
objects.add(new DeleteObject(s));
try {
removeFile(bucketName, s);
} catch (Exception e) {
log.error("[MinioUtil]>>>>batch delete file,Exception:", e);
}
});
}
/**
* get file url
*
* @param bucketName
* @param objectName
* @param expires
* @return url
*/
@SneakyThrows(Exception.class)
public String getPresignedObjectUrl(String bucketName, String objectName, Integer expires) {
GetPresignedObjectUrlArgs args = GetPresignedObjectUrlArgs.builder().expiry(expires).bucket(bucketName).object(objectName).build();
return minioClient.getPresignedObjectUrl(args);
}
/**
* get file url
*
* @param bucketName
* @param objectName
* @return url
*/
@SneakyThrows(Exception.class)
public String getPresignedObjectUrl(String bucketName, String objectName) {
GetPresignedObjectUrlArgs args = GetPresignedObjectUrlArgs.builder()
.bucket(bucketName)
.object(objectName)
.method(Method.GET).build();
return minioClient.getPresignedObjectUrl(args);
}
/**
* change URLDecoder to UTF8
*
* @param str
* @return
* @throws UnsupportedEncodingException
*/
public String getUtf8ByURLDecoder(String str) throws UnsupportedEncodingException {
String url = str.replaceAll("%(?![0-9a-fA-F]{2})", "%25");
return URLDecoder.decode(url, "UTF-8");
}
}
4 测试
5 代码仓库
https://github.com/363153421/springboot-demo/tree/master/minio