MinIO存储桶通知 - Kafka小测

概述

公司的某个项目需要用上这玩意,所以在本地搭建测试环境,经过一番折腾,测试通过,博文记录,用以备忘

MinIO安装

该节不做说明,网络有很多现成的帖子,自行搜索去

配置步骤

控制台添加事件通知



将具体存储桶和事件通知关联起来

点击"Save"保存即可。

注意:

  1. evtest是我预先创建的存储桶,具体你们根据实际场景变更
  2. 填写了桶文件的前后缀,当桶路径匹配不上的时候,不会发送事件通知

事件通知内容样例

PUT

bash 复制代码
{
	"EventName": "s3:ObjectCreated:Put",
	"Key": "evtest/111.conf",
	"Records": [
		{
			"eventVersion": "2.0",
			"eventSource": "minio:s3",
			"awsRegion": "",
			"eventTime": "2024-07-22T01:31:23.190Z",
			"eventName": "s3:ObjectCreated:Put",
			"userIdentity": {
				"principalId": "admin"
			},
			"requestParameters": {
				"principalId": "admin",
				"region": "",
				"sourceIPAddress": "192.168.64.1"
			},
			"responseElements": {
				"x-amz-id-2": "dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8",
				"x-amz-request-id": "17E464E2E26B8A25",
				"x-minio-deployment-id": "3f95da32-9ae8-4983-9622-df88df89410d",
				"x-minio-origin-endpoint": "http://192.168.64.136:9000"
			},
			"s3": {
				"s3SchemaVersion": "1.0",
				"configurationId": "Config",
				"bucket": {
					"name": "evtest",
					"ownerIdentity": {
						"principalId": "admin"
					},
					"arn": "arn:aws:s3:::evtest"
				},
				"object": {
					"key": "111.conf",
					"size": 4763,
					"eTag": "15c4659f418b62e001f7d597dedaf51c",
					"contentType": "application/octet-stream",
					"userMetadata": {
						"content-type": "application/octet-stream"
					},
					"sequencer": "17E464E2E272AAA7"
				}
			},
			"source": {
				"host": "192.168.64.1",
				"port": "",
				"userAgent": "MinIO (linux; amd64) minio-go/v7.0.73 MinIO Console/(dev)"
			}
		}
	]
}

DELETE

bash 复制代码
{
	"EventName": "s3:ObjectRemoved:Delete",
	"Key": "evtest/111.conf",
	"Records": [
		{
			"eventVersion": "2.0",
			"eventSource": "minio:s3",
			"awsRegion": "",
			"eventTime": "2024-07-22T01:56:53.672Z",
			"eventName": "s3:ObjectRemoved:Delete",
			"userIdentity": {
				"principalId": "admin"
			},
			"requestParameters": {
				"principalId": "admin",
				"region": "",
				"sourceIPAddress": "192.168.64.1"
			},
			"responseElements": {
				"content-length": "154",
				"x-amz-id-2": "dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8",
				"x-amz-request-id": "17E466473A6E11BB",
				"x-minio-deployment-id": "3f95da32-9ae8-4983-9622-df88df89410d",
				"x-minio-origin-endpoint": "http://192.168.64.136:9000"
			},
			"s3": {
				"s3SchemaVersion": "1.0",
				"configurationId": "Config",
				"bucket": {
					"name": "evtest",
					"ownerIdentity": {
						"principalId": "admin"
					},
					"arn": "arn:aws:s3:::evtest"
				},
				"object": {
					"key": "111.conf",
					"sequencer": "17E466473A7E072B"
				}
			},
			"source": {
				"host": "192.168.64.1",
				"port": "",
				"userAgent": "MinIO (linux; amd64) minio-go/v7.0.73 MinIO Console/(dev)"
			}
		}
	]
}
bash 复制代码
{
	"EventName": "s3:ObjectAccessed:Head",
	"Key": "evtest/111.conf",
	"Records": [
		{
			"eventVersion": "2.0",
			"eventSource": "minio:s3",
			"awsRegion": "",
			"eventTime": "2024-07-22T01:56:46.230Z",
			"eventName": "s3:ObjectAccessed:Head",
			"userIdentity": {
				"principalId": "admin"
			},
			"requestParameters": {
				"principalId": "admin",
				"region": "",
				"sourceIPAddress": "192.168.64.1"
			},
			"responseElements": {
				"content-length": "4763",
				"x-amz-id-2": "dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8",
				"x-amz-request-id": "17E466457ED88843",
				"x-minio-deployment-id": "3f95da32-9ae8-4983-9622-df88df89410d",
				"x-minio-origin-endpoint": "http://192.168.64.136:9000"
			},
			"s3": {
				"s3SchemaVersion": "1.0",
				"configurationId": "Config",
				"bucket": {
					"name": "evtest",
					"ownerIdentity": {
						"principalId": "admin"
					},
					"arn": "arn:aws:s3:::evtest"
				},
				"object": {
					"key": "111.conf",
					"size": 4763,
					"eTag": "15c4659f418b62e001f7d597dedaf51c",
					"contentType": "application/octet-stream",
					"userMetadata": {
						"content-type": "application/octet-stream"
					},
					"sequencer": "17E464E2E272AAA7"
				}
			},
			"source": {
				"host": "192.168.64.1",
				"port": "",
				"userAgent": "MinIO (linux; amd64) minio-go/v7.0.73 MinIO Console/(dev)"
			}
		}
	]
}

以上通知样例都是通过MinIO Console操作,然后从Kafka接收到的具体内容

参考链接

Minio Bucket 通知试用&&说明
MinIO存储桶通知指南
发布事件到Kafka

相关推荐
小马爱打代码8 小时前
SpringBoot原生实现分布式MapReduce计算
spring boot·分布式·mapreduce
南客先生9 小时前
互联网大厂Java面试:RocketMQ、RabbitMQ与Kafka的深度解析
java·面试·kafka·rabbitmq·rocketmq·消息中间件
悻运9 小时前
如何配置Spark
大数据·分布式·spark
懒惰的橘猫9 小时前
Spark集群搭建之Yarn模式
大数据·分布式·spark
2401_824256869 小时前
Spark-Streaming
大数据·分布式·spark
知其_所以然10 小时前
如何使用@KafkaListener实现从nacos中动态获取监听的topic
kafka
淋一遍下雨天11 小时前
Spark-Streaming核心编程
大数据·kafka
爱吃泡芙的小白白14 小时前
爬虫学习——使用HTTP服务代理、redis使用、通过Scrapy实现分布式爬取
redis·分布式·爬虫·http代理·学习记录
樟小叶_公众号同名14 小时前
Kafka运维经验
后端·kafka
小名叫咸菜15 小时前
flume整合Kafka和spark-streaming核心编程
kafka·flume