Java开发:Spring Cloud Alibaba微服务之消息队列(RocketMQ、Kafka、RabbitMQ)
在Spring Cloud Alibaba 微服务在拆分业务、提升迭代效率的同时,也带来了服务间强依赖、同步阻塞、流量冲击、数据一致性等一系列挑战。而消息队列,正是打通服务异步通信、实现应用解耦、保障系统弹性的关键中间件。
RocketMQ、Kafka、RabbitMQ 作为当前业界应用最广泛的三大消息引擎,在高并发、高可用、高可靠的分布式场景中承担着核心支撑作用。它们以不同的设计理念与性能特性,适配交易支付、日志采集、数据流转、事件驱动、削峰填谷等多样化业务需求。
本文将立足于 Java 微服务开发实践,围绕 Spring Cloud Alibaba 技术栈,对三大消息队列进行场景化解析,帮助开发者理解其设计思想、技术差异与落地选型,为构建稳定、高效、可扩展的分布式系统提供理论支撑与实践指引。
消息队列总体架构
#mermaid-svg-KLTRxnwXn2qtjc2L{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-KLTRxnwXn2qtjc2L .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-KLTRxnwXn2qtjc2L .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-KLTRxnwXn2qtjc2L .error-icon{fill:#552222;}#mermaid-svg-KLTRxnwXn2qtjc2L .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-KLTRxnwXn2qtjc2L .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-KLTRxnwXn2qtjc2L .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-KLTRxnwXn2qtjc2L .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-KLTRxnwXn2qtjc2L .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-KLTRxnwXn2qtjc2L .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-KLTRxnwXn2qtjc2L .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-KLTRxnwXn2qtjc2L .marker{fill:#333333;stroke:#333333;}#mermaid-svg-KLTRxnwXn2qtjc2L .marker.cross{stroke:#333333;}#mermaid-svg-KLTRxnwXn2qtjc2L svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-KLTRxnwXn2qtjc2L p{margin:0;}#mermaid-svg-KLTRxnwXn2qtjc2L .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-KLTRxnwXn2qtjc2L .cluster-label text{fill:#333;}#mermaid-svg-KLTRxnwXn2qtjc2L .cluster-label span{color:#333;}#mermaid-svg-KLTRxnwXn2qtjc2L .cluster-label span p{background-color:transparent;}#mermaid-svg-KLTRxnwXn2qtjc2L .label text,#mermaid-svg-KLTRxnwXn2qtjc2L span{fill:#333;color:#333;}#mermaid-svg-KLTRxnwXn2qtjc2L .node rect,#mermaid-svg-KLTRxnwXn2qtjc2L .node circle,#mermaid-svg-KLTRxnwXn2qtjc2L .node ellipse,#mermaid-svg-KLTRxnwXn2qtjc2L .node polygon,#mermaid-svg-KLTRxnwXn2qtjc2L .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-KLTRxnwXn2qtjc2L .rough-node .label text,#mermaid-svg-KLTRxnwXn2qtjc2L .node .label text,#mermaid-svg-KLTRxnwXn2qtjc2L .image-shape .label,#mermaid-svg-KLTRxnwXn2qtjc2L .icon-shape .label{text-anchor:middle;}#mermaid-svg-KLTRxnwXn2qtjc2L .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-KLTRxnwXn2qtjc2L .rough-node .label,#mermaid-svg-KLTRxnwXn2qtjc2L .node .label,#mermaid-svg-KLTRxnwXn2qtjc2L .image-shape .label,#mermaid-svg-KLTRxnwXn2qtjc2L .icon-shape .label{text-align:center;}#mermaid-svg-KLTRxnwXn2qtjc2L .node.clickable{cursor:pointer;}#mermaid-svg-KLTRxnwXn2qtjc2L .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-KLTRxnwXn2qtjc2L .arrowheadPath{fill:#333333;}#mermaid-svg-KLTRxnwXn2qtjc2L .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-KLTRxnwXn2qtjc2L .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-KLTRxnwXn2qtjc2L .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-KLTRxnwXn2qtjc2L .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-KLTRxnwXn2qtjc2L .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-KLTRxnwXn2qtjc2L .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-KLTRxnwXn2qtjc2L .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-KLTRxnwXn2qtjc2L .cluster text{fill:#333;}#mermaid-svg-KLTRxnwXn2qtjc2L .cluster span{color:#333;}#mermaid-svg-KLTRxnwXn2qtjc2L div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-KLTRxnwXn2qtjc2L .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-KLTRxnwXn2qtjc2L rect.text{fill:none;stroke-width:0;}#mermaid-svg-KLTRxnwXn2qtjc2L .icon-shape,#mermaid-svg-KLTRxnwXn2qtjc2L .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-KLTRxnwXn2qtjc2L .icon-shape p,#mermaid-svg-KLTRxnwXn2qtjc2L .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-KLTRxnwXn2qtjc2L .icon-shape .label rect,#mermaid-svg-KLTRxnwXn2qtjc2L .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-KLTRxnwXn2qtjc2L .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-KLTRxnwXn2qtjc2L .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-KLTRxnwXn2qtjc2L :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 业务服务 A
订单 / 支付 / 库存
消息生产者
Producer
消息队列中间件
RabbitMQ
灵活路由
RocketMQ
事务 / 延迟消息
Kafka
高吞吐流式数据
消息消费者
Consumer
业务服务 B
通知 / 积分 / 物流
可靠性能力
确认 / 重试 / 死信 / 持久化
一、RabbitMQ(灵活路由,复杂场景)
缓存与队列:RabbitMQ------轻量灵活路由,中小场景首选
Docker实战:RabbitMQ容器化部署,三种姿势
1. 核心信息
- 定位:基于 AMQP 协议的开源消息中间件,核心是灵活的消息路由机制。
- 核心特性:
- 基于交换机(Exchange)/ 队列(Queue)模型,路由规则丰富(直连、主题、扇出、头部);
- 支持消息确认(生产者确认、消费者确认)、死信队列;
- 吞吐量中等(万级 / 秒),但路由灵活性远超其他 MQ;
- 支持延迟消息(需插件)、优先级队列。
核心概念:Exchange(交换机)、Queue(队列)、Binding(绑定)、Routing Key(路由键)、Virtual Host(虚拟主机)。
RabbitMQ 的核心在于"生产者不直接投递到队列",而是先进入交换机,再由交换机按照绑定规则和路由键分发到不同队列。
#mermaid-svg-t4POZ50rBZr5UHnJ{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-t4POZ50rBZr5UHnJ .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-t4POZ50rBZr5UHnJ .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-t4POZ50rBZr5UHnJ .error-icon{fill:#552222;}#mermaid-svg-t4POZ50rBZr5UHnJ .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-t4POZ50rBZr5UHnJ .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-t4POZ50rBZr5UHnJ .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-t4POZ50rBZr5UHnJ .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-t4POZ50rBZr5UHnJ .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-t4POZ50rBZr5UHnJ .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-t4POZ50rBZr5UHnJ .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-t4POZ50rBZr5UHnJ .marker{fill:#333333;stroke:#333333;}#mermaid-svg-t4POZ50rBZr5UHnJ .marker.cross{stroke:#333333;}#mermaid-svg-t4POZ50rBZr5UHnJ svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-t4POZ50rBZr5UHnJ p{margin:0;}#mermaid-svg-t4POZ50rBZr5UHnJ .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-t4POZ50rBZr5UHnJ .cluster-label text{fill:#333;}#mermaid-svg-t4POZ50rBZr5UHnJ .cluster-label span{color:#333;}#mermaid-svg-t4POZ50rBZr5UHnJ .cluster-label span p{background-color:transparent;}#mermaid-svg-t4POZ50rBZr5UHnJ .label text,#mermaid-svg-t4POZ50rBZr5UHnJ span{fill:#333;color:#333;}#mermaid-svg-t4POZ50rBZr5UHnJ .node rect,#mermaid-svg-t4POZ50rBZr5UHnJ .node circle,#mermaid-svg-t4POZ50rBZr5UHnJ .node ellipse,#mermaid-svg-t4POZ50rBZr5UHnJ .node polygon,#mermaid-svg-t4POZ50rBZr5UHnJ .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-t4POZ50rBZr5UHnJ .rough-node .label text,#mermaid-svg-t4POZ50rBZr5UHnJ .node .label text,#mermaid-svg-t4POZ50rBZr5UHnJ .image-shape .label,#mermaid-svg-t4POZ50rBZr5UHnJ .icon-shape .label{text-anchor:middle;}#mermaid-svg-t4POZ50rBZr5UHnJ .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-t4POZ50rBZr5UHnJ .rough-node .label,#mermaid-svg-t4POZ50rBZr5UHnJ .node .label,#mermaid-svg-t4POZ50rBZr5UHnJ .image-shape .label,#mermaid-svg-t4POZ50rBZr5UHnJ .icon-shape .label{text-align:center;}#mermaid-svg-t4POZ50rBZr5UHnJ .node.clickable{cursor:pointer;}#mermaid-svg-t4POZ50rBZr5UHnJ .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-t4POZ50rBZr5UHnJ .arrowheadPath{fill:#333333;}#mermaid-svg-t4POZ50rBZr5UHnJ .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-t4POZ50rBZr5UHnJ .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-t4POZ50rBZr5UHnJ .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-t4POZ50rBZr5UHnJ .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-t4POZ50rBZr5UHnJ .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-t4POZ50rBZr5UHnJ .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-t4POZ50rBZr5UHnJ .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-t4POZ50rBZr5UHnJ .cluster text{fill:#333;}#mermaid-svg-t4POZ50rBZr5UHnJ .cluster span{color:#333;}#mermaid-svg-t4POZ50rBZr5UHnJ div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-t4POZ50rBZr5UHnJ .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-t4POZ50rBZr5UHnJ rect.text{fill:none;stroke-width:0;}#mermaid-svg-t4POZ50rBZr5UHnJ .icon-shape,#mermaid-svg-t4POZ50rBZr5UHnJ .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-t4POZ50rBZr5UHnJ .icon-shape p,#mermaid-svg-t4POZ50rBZr5UHnJ .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-t4POZ50rBZr5UHnJ .icon-shape .label rect,#mermaid-svg-t4POZ50rBZr5UHnJ .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-t4POZ50rBZr5UHnJ .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-t4POZ50rBZr5UHnJ .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-t4POZ50rBZr5UHnJ :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} Routing Key = order.create
Routing Key = user.*
Fanout / Topic / Direct
失败消息
Producer
生产者
Exchange
交换机
Queue
订单队列
Queue
用户队列
Queue
通知队列
Consumer
订单服务
Consumer
用户服务
Consumer
通知服务
Dead Letter Queue
死信队列
2. Spring Cloud Alibaba 整合示例
1. 依赖引入(pom.xml)
xml
<!-- Spring AMQP(RabbitMQ 核心) -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
2. 配置文件(application.yml)
yaml
spring:
# RabbitMQ 连接配置
rabbitmq:
host: 192.168.1.18 # RabbitMQ 服务器地址
port: 5672 # 端口
username: admin # 账号
password: 123456 # 密码
virtual-host: /my_vhost # 虚拟主机,根据自己的来
# 生产者确认配置(保证消息可靠投递)
publisher-confirm-type: correlated # 开启生产者确认(异步)
publisher-returns: true # 开启消息返回机制
# 消费者配置
listener:
simple:
acknowledge-mode: manual # 手动确认消息(保证消息不丢失)
concurrency: 1 # 最小消费线程数
max-concurrency: 5 # 最大消费线程数
retry:
enabled: true # 开启消费重试
max-attempts: 3 # 最大重试次数
initial-interval: 1000 # 重试间隔(毫秒)
3. RabbitMQ 队列 / 交换机 / 绑定配置
java
package com.microlink.provider.config;
import org.springframework.amqp.core.*;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
/**
* RabbitMQ 队列、交换机、绑定配置类
*/
@Configuration
public class RabbitMqConfig {
// 1. 定义常量(队列、交换机、路由键)
/** 队列名称 */
public static final String DEMO_QUEUE = "demo_queue";
/** 交换机名称(直连交换机) */
public static final String DEMO_EXCHANGE = "demo_exchange";
/** 路由键 */
public static final String DEMO_ROUTING_KEY = "demo.routing.key";
// 2. 创建队列
@Bean
public Queue demoQueue() {
// 参数说明:
// name: 队列名
// durable: 是否持久化(重启 RabbitMQ 后队列不丢失)
// exclusive: 是否排他(仅当前连接可用)
// autoDelete: 是否自动删除(无消费者时删除)
// arguments: 额外参数(如消息过期时间、死信队列等)
return QueueBuilder.durable(DEMO_QUEUE)
.build();
}
// 3. 创建交换机(直连交换机,精准路由)
@Bean
public DirectExchange demoExchange() {
// durable: 是否持久化
return ExchangeBuilder.directExchange(DEMO_EXCHANGE)
.durable(true)
.build();
}
// 4. 绑定队列和交换机(指定路由键)
@Bean
public Binding demoBinding(Queue demoQueue, DirectExchange demoExchange) {
return BindingBuilder.bind(demoQueue)
.to(demoExchange)
.with(DEMO_ROUTING_KEY);
}
}
4. 生产者实现(发送消息)
java
package com.microlink.provider.component;
import com.microlink.provider.config.RabbitMqConfig;
import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.rabbit.connection.CorrelationData;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import javax.annotation.PostConstruct;
import java.util.UUID;
/**
* RabbitMQ 生产者
*/
@Slf4j
@Component
public class RabbitMqProducer {
@Autowired
private RabbitTemplate rabbitTemplate;
/**
* 初始化生产者确认回调
*/
@PostConstruct
public void init() {
// 1. 消息发送到交换机确认回调
rabbitTemplate.setConfirmCallback((correlationData, ack, cause) -> {
String msgId = correlationData != null ? correlationData.getId() : "未知ID";
if (ack) {
log.info("消息[{}]成功发送到交换机", msgId);
} else {
log.error("消息[{}]发送到交换机失败,原因:{}", msgId, cause);
// 此处可添加消息重试、入库等补偿逻辑
}
});
// 2. 消息无法路由到队列的回调(需要开启 publisher-returns: true)
rabbitTemplate.setReturnCallback((message, replyCode, replyText, exchange, routingKey) -> {
log.error("消息路由失败:交换机={}, 路由键={}, 响应码={}, 响应信息={}, 消息内容={}",
exchange, routingKey, replyCode, replyText, new String(message.getBody()));
});
}
/**
* 发送消息
* @param message 消息内容
*/
public void sendMessage(String message) {
// 生成唯一消息ID(用于追踪)
String msgId = UUID.randomUUID().toString();
CorrelationData correlationData = new CorrelationData(msgId);
// 发送消息:交换机 + 路由键 + 消息内容 + 消息ID
rabbitTemplate.convertAndSend(
RabbitMqConfig.DEMO_EXCHANGE,
RabbitMqConfig.DEMO_ROUTING_KEY,
message,
correlationData
);
log.info("消息[{}]已发送,内容:{}", msgId, message);
}
}
5. 消费者实现(接收消息)
java
package com.microlink.provider.component;
import com.microlink.provider.config.RabbitMqConfig;
import com.rabbitmq.client.Channel;
import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.stereotype.Component;
import java.io.IOException;
/**
* RabbitMQ 消费者
*/
@Slf4j
@Component
public class RabbitMqConsumer {
/**
* 监听指定队列
* @param message 消息内容
* @param channel 信道
* @param msg 原始消息对象
* @throws IOException IO异常
*/
@RabbitListener(queues = RabbitMqConfig.DEMO_QUEUE)
public void consumeMessage(String message, Channel channel, Message msg) throws IOException {
// 获取消息的投递标签(唯一标识)
long deliveryTag = msg.getMessageProperties().getDeliveryTag();
try {
// 1. 处理业务逻辑
log.info("收到消息,deliveryTag={}, 内容:{}", deliveryTag, message);
// 模拟业务处理(如:保存到数据库、调用接口等)
// TODO: 替换为你的实际业务逻辑
handleBusinessLogic(message);
// 2. 手动确认消息(成功处理,告知RabbitMQ删除消息)
// multiple: false 表示只确认当前消息,true 表示确认所有小于等于当前deliveryTag的消息
channel.basicAck(deliveryTag, false);
log.info("消息[{}]确认成功", deliveryTag);
} catch (Exception e) {
log.error("处理消息失败,deliveryTag={}, 内容:{},异常:{}", deliveryTag, message, e.getMessage(), e);
// 3. 处理失败:拒绝消息并重新入队(重试),或直接拒绝(进入死信队列)
// requeue: true 重新入队,false 不重新入队(需配置死信队列)
channel.basicNack(deliveryTag, false, true);
log.info("消息[{}]处理失败,已拒绝并重新入队", deliveryTag);
}
}
/**
* 模拟业务逻辑处理
* @param message 消息内容
*/
private void handleBusinessLogic(String message) {
// 此处编写你的实际业务逻辑
log.info("业务逻辑处理完成:{}", message);
}
}
6. 测试接口(发送消息)
java
package com.microlink.provider.controller;
import com.microlink.provider.component.RabbitMqProducer;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import javax.annotation.Resource;
/**
* @author rh
* @date 2026/3/15 14:52
* @ description
*/
@RestController
public class RabbitMQProducerController {
@Resource
private RabbitMqProducer rabbitMqProducer;
/**
* 发送消息接口
* @param message 消息内容
* @return 响应结果
*/
@GetMapping("/send")
public String sendMessage(@RequestParam(defaultValue = "Hello RabbitMQ!") String message) {
rabbitMqProducer.sendMessage(message);
return "消息发送成功,内容:" + message;
}
}
3. 适用场景
- 消息路由复杂的场景(如按用户类型 / 行为分发消息);
- 跨系统协议互通(如与外部系统基于 AMQP 协议对接);
- 需要优先级队列、死信队列的场景(如订单优先级处理);
- 中小型系统的通用消息场景(对吞吐量要求不高)。
二、RocketMQ(阿里原生,微服务首选)
缓存与队列:RocketMQ------消息中间件,企业级业务首选
Docker实战:RocketMQ容器化部署,三种姿势
1. 核心信息
- 定位:阿里开源的分布式消息中间件,专为高可用、高可靠的分布式系统设计,是 Spring Cloud Alibaba 生态的核心组件。
- 核心特性:
- 支持事务消息(解决分布式事务问题)、延迟消息(定时任务场景);
- 基于主题/ 队列模型,支持集群消费、广播消费;
- 消息持久化(磁盘存储),支持消息回溯、死信队列;
- 吞吐量高(10 万 +/ 秒),性能介于 Kafka 和 RabbitMQ 之间。
核心概念:NameServer(服务发现)、Broker(消息存储 / 转发)、Producer(生产者)、Consumer(消费者)、Topic(主题)、Group(消费组)。
RocketMQ 更强调企业级业务可靠性,典型链路是生产者通过 NameServer 发现 Broker,再将消息写入 Topic,消费者按消费组拉取或推送消费。
#mermaid-svg-gBaitdDOTMpkoaYo{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-gBaitdDOTMpkoaYo .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-gBaitdDOTMpkoaYo .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-gBaitdDOTMpkoaYo .error-icon{fill:#552222;}#mermaid-svg-gBaitdDOTMpkoaYo .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-gBaitdDOTMpkoaYo .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-gBaitdDOTMpkoaYo .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-gBaitdDOTMpkoaYo .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-gBaitdDOTMpkoaYo .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-gBaitdDOTMpkoaYo .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-gBaitdDOTMpkoaYo .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-gBaitdDOTMpkoaYo .marker{fill:#333333;stroke:#333333;}#mermaid-svg-gBaitdDOTMpkoaYo .marker.cross{stroke:#333333;}#mermaid-svg-gBaitdDOTMpkoaYo svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-gBaitdDOTMpkoaYo p{margin:0;}#mermaid-svg-gBaitdDOTMpkoaYo .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-gBaitdDOTMpkoaYo .cluster-label text{fill:#333;}#mermaid-svg-gBaitdDOTMpkoaYo .cluster-label span{color:#333;}#mermaid-svg-gBaitdDOTMpkoaYo .cluster-label span p{background-color:transparent;}#mermaid-svg-gBaitdDOTMpkoaYo .label text,#mermaid-svg-gBaitdDOTMpkoaYo span{fill:#333;color:#333;}#mermaid-svg-gBaitdDOTMpkoaYo .node rect,#mermaid-svg-gBaitdDOTMpkoaYo .node circle,#mermaid-svg-gBaitdDOTMpkoaYo .node ellipse,#mermaid-svg-gBaitdDOTMpkoaYo .node polygon,#mermaid-svg-gBaitdDOTMpkoaYo .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-gBaitdDOTMpkoaYo .rough-node .label text,#mermaid-svg-gBaitdDOTMpkoaYo .node .label text,#mermaid-svg-gBaitdDOTMpkoaYo .image-shape .label,#mermaid-svg-gBaitdDOTMpkoaYo .icon-shape .label{text-anchor:middle;}#mermaid-svg-gBaitdDOTMpkoaYo .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-gBaitdDOTMpkoaYo .rough-node .label,#mermaid-svg-gBaitdDOTMpkoaYo .node .label,#mermaid-svg-gBaitdDOTMpkoaYo .image-shape .label,#mermaid-svg-gBaitdDOTMpkoaYo .icon-shape .label{text-align:center;}#mermaid-svg-gBaitdDOTMpkoaYo .node.clickable{cursor:pointer;}#mermaid-svg-gBaitdDOTMpkoaYo .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-gBaitdDOTMpkoaYo .arrowheadPath{fill:#333333;}#mermaid-svg-gBaitdDOTMpkoaYo .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-gBaitdDOTMpkoaYo .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-gBaitdDOTMpkoaYo .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-gBaitdDOTMpkoaYo .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-gBaitdDOTMpkoaYo .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-gBaitdDOTMpkoaYo .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-gBaitdDOTMpkoaYo .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-gBaitdDOTMpkoaYo .cluster text{fill:#333;}#mermaid-svg-gBaitdDOTMpkoaYo .cluster span{color:#333;}#mermaid-svg-gBaitdDOTMpkoaYo div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-gBaitdDOTMpkoaYo .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-gBaitdDOTMpkoaYo rect.text{fill:none;stroke-width:0;}#mermaid-svg-gBaitdDOTMpkoaYo .icon-shape,#mermaid-svg-gBaitdDOTMpkoaYo .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-gBaitdDOTMpkoaYo .icon-shape p,#mermaid-svg-gBaitdDOTMpkoaYo .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-gBaitdDOTMpkoaYo .icon-shape .label rect,#mermaid-svg-gBaitdDOTMpkoaYo .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-gBaitdDOTMpkoaYo .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-gBaitdDOTMpkoaYo .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-gBaitdDOTMpkoaYo :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 查询路由
注册路由
主从复制
发送消息
NameServer
路由注册与发现
Producer Group
生产者组
Broker Master
消息写入 / 转发
Broker Slave
数据复制 / 高可用
Topic
业务主题
MessageQueue 1
MessageQueue 2
Consumer Group
消费者组
Consumer 实例 A
Consumer 实例 B
2. Spring Cloud Alibaba 整合示例
1. 依赖引入(pom.xml)
xml
<!-- RocketMQ 原生客户端(用于手动创建主题) -->
<dependency>
<groupId>org.apache.rocketmq</groupId>
<artifactId>rocketmq-client</artifactId>
<version>4.9.4</version> <!-- 和你的 rocketmq-client 版本保持一致 -->
</dependency>
2. 配置文件(application.yml)
yaml
# RocketMQ 配置
rocketmq:
# NameServer 地址(多个用;分隔)
name-server: 192.168.1.18:9876
# 生产者配置
producer:
# 生产者组名
group: demo-producer-group
# 发送超时时间(毫秒)
send-message-timeout: 3000
# 重试次数
retry-times-when-send-failed: 2
# 消费者配置
consumer:
# 消费者组名
group: demo-consumer-group
# 消费线程数
consume-thread-min: 5
consume-thread-max: 10
3. RocketMQ 配置类(初始化客户端)
java
package com.microlink.provider.config;
import lombok.extern.slf4j.Slf4j;
import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import org.apache.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.common.protocol.heartbeat.MessageModel;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
/**
* RocketMQ配置类
*/
@Configuration
@Slf4j
public class RocketMQConfig {
@Value("${rocketmq.name-server}")
private String nameServer;
@Value("${rocketmq.producer.group}")
private String producerGroup;
@Value("${rocketmq.producer.send-message-timeout}")
private int sendTimeout;
@Value("${rocketmq.consumer.group}")
private String consumerGroup;
@Value("${rocketmq.consumer.consume-thread-min}")
private int consumeThreadMin;
@Value("${rocketmq.consumer.consume-thread-max}")
private int consumeThreadMax;
@Bean(destroyMethod = "shutdown")
public DefaultMQProducer defaultMQProducer() {
DefaultMQProducer producer = new DefaultMQProducer(producerGroup);
producer.setNamesrvAddr(nameServer);
producer.setSendMsgTimeout(sendTimeout);
producer.setRetryTimesWhenSendFailed(2);
try {
producer.start();
log.info("RocketMQ 生产者启动成功!group: {}, nameServer: {}", producerGroup, nameServer);
} catch (Exception e) {
log.error("RocketMQ 生产者启动失败!", e);
throw new RuntimeException("生产者初始化失败", e);
}
return producer;
}
@Bean(destroyMethod = "shutdown")
public DefaultMQPushConsumer defaultMQPushConsumer() {
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer(consumerGroup);
consumer.setNamesrvAddr(nameServer);
consumer.setConsumeThreadMin(consumeThreadMin);
consumer.setConsumeThreadMax(consumeThreadMax);
consumer.setMessageModel(MessageModel.CLUSTERING);
// 关键:4.9.4 版本中 setConsumeFromWhere 已过时,直接移除(默认就是 CONSUME_FROM_LAST_OFFSET)
// 如需设置,可使用字符串常量:consumer.setConsumeFromWhere("CONSUME_FROM_LAST_OFFSET");
try {
consumer.subscribe("demo_topic", "*");
consumer.registerMessageListener((MessageListenerConcurrently) (msgs, context) -> {
try {
for (org.apache.rocketmq.common.message.MessageExt msg : msgs) {
String topic = msg.getTopic();
String tags = msg.getTags();
String body = new String(msg.getBody(), "UTF-8");
log.info("消费消息成功 -> topic: {}, tags: {}, body: {}", topic, tags, body);
}
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
} catch (Exception e) {
log.error("消费消息失败", e);
return ConsumeConcurrentlyStatus.RECONSUME_LATER;
}
});
consumer.start();
log.info("RocketMQ 消费者启动成功!group: {}, nameServer: {}", consumerGroup, nameServer);
} catch (Exception e) {
log.error("RocketMQ 消费者启动失败!", e);
throw new RuntimeException("消费者初始化失败", e);
}
return consumer;
}
}
4. 生产者工具类(发送消息)
java
package com.microlink.provider.component;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.client.producer.SendResult;
import org.apache.rocketmq.common.message.Message;
import org.springframework.stereotype.Component;
/**
* RocketMQ 生产者工具类
*/
@Component
@Slf4j
@RequiredArgsConstructor
public class RocketMQProducer {
private final DefaultMQProducer defaultMQProducer;
/**
* 发送普通消息
* @param topic 主题
* @param tags 标签
* @param body 消息体
*/
public void sendMessage(String topic, String tags, String body) {
try {
// 构建消息
Message message = new Message(
topic, // 主题
tags, // 标签
body.getBytes("UTF-8") // 消息体
);
// 发送消息
SendResult sendResult = defaultMQProducer.send(message);
log.info("发送消息成功 -> 消息ID: {}, 发送状态: {}", sendResult.getMsgId(), sendResult.getSendStatus());
} catch (Exception e) {
log.error("发送消息失败", e);
throw new RuntimeException("消息发送失败", e);
}
}
}
5. 测试接口(触发消息发送)
java
package com.microlink.provider.controller;
import com.microlink.provider.component.RocketMQProducer;
import lombok.RequiredArgsConstructor;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
/**
* 基于 RocketMQ 原生 API 的生产者
*/
@RestController
@RequiredArgsConstructor
public class RocketMQProducerController {
private final RocketMQProducer rocketMQProducer;
/**
* 发送消息接口
* @param body 消息内容
*/
@GetMapping("/sendMessage")
public String sendMessage(@RequestParam(defaultValue = "Hello RocketMQ!") String body) {
// 发送消息到 demo_topic 主题,标签为 demo_tag
rocketMQProducer.sendMessage("demo_topic", "demo_tag", body);
return "消息发送成功!内容:" + body;
}
}
3. 适用场景
- Spring Cloud Alibaba 微服务核心场景(订单创建、库存扣减、支付通知);
- 需要分布式事务保障的场景(如电商下单的事务消息);
- 需要延迟消息的场景(订单超时关闭、优惠券过期提醒);
- 对消息可靠性要求高的金融、电商领域。
三、Kafka(高吞吐,大数据场景)
缓存与队列:Kafka------聚焦高吞吐核心,大数据场景首选
Docker实战:Kafka容器化部署,三种姿势
1. 核心信息
- 定位:LinkedIn 开源的分布式流处理平台,核心是高吞吐、低延迟的消息传递。
- 核心特性:
- 极高吞吐量(百万 +/ 秒),适合海量数据传输;
- 基于分区 / 副本模型,水平扩展能力强;
- 消息持久化(日志文件),支持消息回溯;
- 弱事务支持,更侧重数据传输而非可靠性保障。
核心概念:Broker(服务节点)、Topic(主题)、Partition(分区)、Consumer Group(消费组)、Offset(偏移量)。
Kafka 的核心是"Topic + Partition + Offset"。生产者按 Key 或分区策略写入分区,消费者组内的实例共同消费不同分区,以换取高吞吐和水平扩展能力。
#mermaid-svg-DVZ4oWp40iFQwyor{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-DVZ4oWp40iFQwyor .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-DVZ4oWp40iFQwyor .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-DVZ4oWp40iFQwyor .error-icon{fill:#552222;}#mermaid-svg-DVZ4oWp40iFQwyor .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-DVZ4oWp40iFQwyor .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-DVZ4oWp40iFQwyor .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-DVZ4oWp40iFQwyor .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-DVZ4oWp40iFQwyor .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-DVZ4oWp40iFQwyor .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-DVZ4oWp40iFQwyor .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-DVZ4oWp40iFQwyor .marker{fill:#333333;stroke:#333333;}#mermaid-svg-DVZ4oWp40iFQwyor .marker.cross{stroke:#333333;}#mermaid-svg-DVZ4oWp40iFQwyor svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-DVZ4oWp40iFQwyor p{margin:0;}#mermaid-svg-DVZ4oWp40iFQwyor .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-DVZ4oWp40iFQwyor .cluster-label text{fill:#333;}#mermaid-svg-DVZ4oWp40iFQwyor .cluster-label span{color:#333;}#mermaid-svg-DVZ4oWp40iFQwyor .cluster-label span p{background-color:transparent;}#mermaid-svg-DVZ4oWp40iFQwyor .label text,#mermaid-svg-DVZ4oWp40iFQwyor span{fill:#333;color:#333;}#mermaid-svg-DVZ4oWp40iFQwyor .node rect,#mermaid-svg-DVZ4oWp40iFQwyor .node circle,#mermaid-svg-DVZ4oWp40iFQwyor .node ellipse,#mermaid-svg-DVZ4oWp40iFQwyor .node polygon,#mermaid-svg-DVZ4oWp40iFQwyor .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-DVZ4oWp40iFQwyor .rough-node .label text,#mermaid-svg-DVZ4oWp40iFQwyor .node .label text,#mermaid-svg-DVZ4oWp40iFQwyor .image-shape .label,#mermaid-svg-DVZ4oWp40iFQwyor .icon-shape .label{text-anchor:middle;}#mermaid-svg-DVZ4oWp40iFQwyor .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-DVZ4oWp40iFQwyor .rough-node .label,#mermaid-svg-DVZ4oWp40iFQwyor .node .label,#mermaid-svg-DVZ4oWp40iFQwyor .image-shape .label,#mermaid-svg-DVZ4oWp40iFQwyor .icon-shape .label{text-align:center;}#mermaid-svg-DVZ4oWp40iFQwyor .node.clickable{cursor:pointer;}#mermaid-svg-DVZ4oWp40iFQwyor .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-DVZ4oWp40iFQwyor .arrowheadPath{fill:#333333;}#mermaid-svg-DVZ4oWp40iFQwyor .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-DVZ4oWp40iFQwyor .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-DVZ4oWp40iFQwyor .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-DVZ4oWp40iFQwyor .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-DVZ4oWp40iFQwyor .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-DVZ4oWp40iFQwyor .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-DVZ4oWp40iFQwyor .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-DVZ4oWp40iFQwyor .cluster text{fill:#333;}#mermaid-svg-DVZ4oWp40iFQwyor .cluster span{color:#333;}#mermaid-svg-DVZ4oWp40iFQwyor div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-DVZ4oWp40iFQwyor .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-DVZ4oWp40iFQwyor rect.text{fill:none;stroke-width:0;}#mermaid-svg-DVZ4oWp40iFQwyor .icon-shape,#mermaid-svg-DVZ4oWp40iFQwyor .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-DVZ4oWp40iFQwyor .icon-shape p,#mermaid-svg-DVZ4oWp40iFQwyor .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-DVZ4oWp40iFQwyor .icon-shape .label rect,#mermaid-svg-DVZ4oWp40iFQwyor .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-DVZ4oWp40iFQwyor .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-DVZ4oWp40iFQwyor .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-DVZ4oWp40iFQwyor :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} key=user-1
key=user-2
Producer A
Topic
test_topic
Producer B
Partition 0
offset 0..n
Partition 1
offset 0..n
Partition 2
offset 0..n
Consumer A
Group: kafka-demo
Consumer B
Group: kafka-demo
Consumer C
Group: kafka-demo
提交 Offset
提交 Offset
提交 Offset
2. Spring Cloud Alibaba 整合示例
1. 依赖引入(pom.xml)
xml
<!-- Kafka 原生客户端(Spring 封装) -->
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka</artifactId>
</dependency>
2. 配置文件(application.yml)
yaml
spring:
kafka:
bootstrap-servers: 192.168.1.18:9092
consumer:
group-id: test-group-100
auto-offset-reset: earliest
key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
value-deserializer: org.apache.kafka.common.serialization.StringDeserializer
listener:
missing-topics-fatal: false
# 强制消费者容器立即启动
consumer-property:
session.timeout.ms: 30000
heartbeat.interval.ms: 10000
3. Kafka原生客户端配置类
java
package com.microlink.provider.config;
import org.apache.kafka.clients.consumer.ConsumerConfig;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.common.serialization.StringDeserializer;
import org.apache.kafka.common.serialization.StringSerializer;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.kafka.annotation.EnableKafka;
import org.springframework.kafka.config.ConcurrentKafkaListenerContainerFactory;
import org.springframework.kafka.config.KafkaListenerContainerFactory;
import org.springframework.kafka.core.*;
import org.springframework.kafka.listener.ConcurrentMessageListenerContainer;
import java.util.HashMap;
import java.util.Map;
/**
* Kafka 原生客户端配置类
*/
@Configuration
@EnableKafka // 开启 Kafka 监听
public class KafkaConfig {
// 从配置文件读取 Kafka 服务器地址
@Value("${spring.kafka.bootstrap-servers}")
private String bootstrapServers;
// 从配置文件读取消费组 ID
@Value("${spring.kafka.consumer.group-id}")
private String groupId;
/**
* 生产者配置参数
*/
public Map<String, Object> producerConfigs() {
Map<String, Object> props = new HashMap<>();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
props.put(ProducerConfig.RETRIES_CONFIG, 3); // 重试次数
props.put(ProducerConfig.BATCH_SIZE_CONFIG, 16384); // 批量大小
props.put(ProducerConfig.BUFFER_MEMORY_CONFIG, 33554432); // 缓冲区大小
props.put(ProducerConfig.ACKS_CONFIG, "1"); // 确认机制
return props;
}
/**
* 创建生产者工厂
*/
@Bean
public ProducerFactory<String, String> producerFactory() {
return new DefaultKafkaProducerFactory<>(producerConfigs());
}
/**
* 创建 KafkaTemplate(简化生产者操作)
*/
@Bean
public KafkaTemplate<String, String> kafkaTemplate() {
return new KafkaTemplate<>(producerFactory());
}
/**
* 消费者配置参数
*/
public Map<String, Object> consumerConfigs() {
Map<String, Object> props = new HashMap<>();
props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
props.put(ConsumerConfig.GROUP_ID_CONFIG, groupId);
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, true); // 自动提交偏移量
props.put(ConsumerConfig.AUTO_COMMIT_INTERVAL_MS_CONFIG, 1000); // 自动提交间隔
props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest"); // 从头消费
return props;
}
/**
* 创建消费者工厂
*/
@Bean
public ConsumerFactory<String, String> consumerFactory() {
return new DefaultKafkaConsumerFactory<>(consumerConfigs());
}
/**
* 创建监听容器工厂(用于 @KafkaListener 注解)
*/
@Bean
public KafkaListenerContainerFactory<ConcurrentMessageListenerContainer<String, String>> kafkaListenerContainerFactory() {
ConcurrentKafkaListenerContainerFactory<String, String> factory = new ConcurrentKafkaListenerContainerFactory<>();
factory.setConsumerFactory(consumerFactory());
factory.setConcurrency(3); // 并发消费线程数
factory.getContainerProperties().setPollTimeout(3000); // 拉取超时时间
return factory;
}
}
4. Kafka 生产者(原生客户端封装)
java
package com.microlink.provider.component;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.support.SendResult;
import org.springframework.stereotype.Component;
import org.springframework.util.concurrent.ListenableFuture;
import org.springframework.util.concurrent.ListenableFutureCallback;
/**
* Kafka 生产者(原生客户端封装)
*/
@Slf4j
@Component
@RequiredArgsConstructor
public class KafkaProducer {
// 注入 KafkaTemplate(简化原生客户端调用)
private final KafkaTemplate<String, String> kafkaTemplate;
/**
* 发送消息(异步)
* @param topic 主题名称
* @param message 消息内容
*/
public void sendMessage(String topic, String message) {
// 异步发送消息
ListenableFuture<SendResult<String, String>> future = kafkaTemplate.send(topic, message);
// 回调处理发送结果
future.addCallback(new ListenableFutureCallback<SendResult<String, String>>() {
@Override
public void onSuccess(SendResult<String, String> result) {
log.info("消息发送成功!topic: {}, message: {}, offset: {}",
topic, message, result.getRecordMetadata().offset());
}
@Override
public void onFailure(Throwable ex) {
log.error("消息发送失败!topic: {}, message: {}", topic, message, ex);
}
});
}
/**
* 发送带 key 的消息(指定分区)
* @param topic 主题名称
* @param key 消息 key(用于分区路由)
* @param message 消息内容
*/
public void sendMessageWithKey(String topic, String key, String message) {
ListenableFuture<SendResult<String, String>> future = kafkaTemplate.send(topic, key, message);
future.addCallback(new ListenableFutureCallback<SendResult<String, String>>() {
@Override
public void onSuccess(SendResult<String, String> result) {
log.info("带Key消息发送成功!topic: {}, key: {}, message: {}, partition: {}",
topic, key, message, result.getRecordMetadata().partition());
}
@Override
public void onFailure(Throwable ex) {
log.error("带Key消息发送失败!topic: {}, key: {}, message: {}", topic, key, message, ex);
}
});
}
}
5. Kafka消费者(原生客户端封装)
java
package com.microlink.provider.component;
import lombok.extern.slf4j.Slf4j;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Component;
import java.util.Optional;
/**
* Kafka 消费者(原生客户端监听)
*/
@Slf4j
@Component
public class KafkaConsumer {
/**
* 监听指定主题的消息
*/
@KafkaListener(
topics = "test_topic",
groupId = "kafka-demo-group-new" // 1. 改全新分组ID,彻底避开旧偏移量
)
public void consume(ConsumerRecord<String, String> record) {
// ========== 核心:用System.out强制打印,绕开log过滤 ==========
System.out.println("=====================================");
System.out.println("===== 消费方法已触发!====="); // 2. 把log.info改成System.out
System.out.println("原始 Record 信息:");
System.out.printf(record.toString());
try {
Optional<String> message = Optional.ofNullable(record.value());
if (message.isPresent()) {
String msg = message.get();
// 控制台强制打印(必现)
System.out.println("接收到消息:topic=" + record.topic() + ", partition=" + record.partition() + ", offset=" + record.offset() + ", key=" + record.key() + ", value=" + msg);
// 双重确认
System.out.println("【控制台】处理消息:" + msg);
System.out.println("=====================================");
} else {
System.out.println("接收到空消息!Record:" + record);
}
} catch (Exception e) {
System.out.println("消费消息时发生异常!" + e);
}
}
/**
* 监听多个主题(备用方法,可保留)
*/
@KafkaListener(
topics = {"test_topic_1", "test_topic_2"},
groupId = "kafka-demo-group-new"
)
public void consumeMultiTopic(ConsumerRecord<String, String> record) {
Optional<String> message = Optional.ofNullable(record.value());
if (message.isPresent()) {
log.info("多主题消费:topic={}, value={}", record.topic(), message.get());
}
}
}
6. 测试接口:发送 Kafka 消息
java
package com.microlink.provider.controller;
import com.microlink.provider.component.KafkaProducer;
import lombok.RequiredArgsConstructor;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
/**
* 测试接口:发送 Kafka 消息
*/
@RestController
@RequestMapping("/kafka")
@RequiredArgsConstructor
public class KafkaProducerController {
private final KafkaProducer kafkaProducer;
/**
* 发送普通消息
* 示例:http://localhost:8080/kafka/send/test_topic/hello-kafka
*/
@GetMapping("/send/{topic}/{message}")
public String sendMessage(@PathVariable String topic, @PathVariable String message) {
kafkaProducer.sendMessage(topic, message);
return "消息发送请求已提交:topic=" + topic + ", message=" + message;
}
/**
* 发送带 Key 的消息
* 示例:http://localhost:8080/kafka/send/key/test_topic/1/hello-key
*/
@GetMapping("/send/key/{topic}/{key}/{message}")
public String sendMessageWithKey(@PathVariable String topic, @PathVariable String key, @PathVariable String message) {
kafkaProducer.sendMessageWithKey(topic, key, message);
return "带Key消息发送请求已提交:topic=" + topic + ", key=" + key + ", message=" + message;
}
}
7. 项目启动类添加注解
java
@EnableKafka
@ComponentScan("com.microlink") // 确保扫描到 KafkaConsumer 类
3. 适用场景
- 海量日志收集(如微服务访问日志、系统监控日志);
- 大数据实时计算(如用户行为分析、实时报表);
- 数据同步 / 迁移(如数据库 binlog 同步);
- 高吞吐、低延迟的消息传输场景(对事务可靠性要求低)
四、Spring Cloud Stream(拓展)
1. 什么是 Spring Cloud Stream?
Spring Cloud Stream 是 Spring 官方推出的消息中间件抽象层,它屏蔽了不同 MQ 的原生 API 差异,让你用一套标准化的代码操作任意 MQ(RocketMQ/Kafka/RabbitMQ),核心目标是 "解耦业务代码与具体 MQ 实现"。
2. 核心设计理念
- 生产者 - 消费者模型:统一定义消息的发送(生产者)和接收(消费者)逻辑;
- 绑定器(Binder):核心抽象层,不同 MQ 对应不同的 Binder(如 RocketMQ Binder、Kafka Binder),业务代码不直接依赖 Binder;
- 通道(Binding):连接业务代码和 Binder 的桥梁,分为输入通道(消费)和输出通道(生产);
- 函数式编程:Spring Cloud Stream 3.0 + 推荐使用函数式编程(Supplier/Consumer/Function)定义消息逻辑,替代传统注解方式。
3. 核心组件与概念
| 组件 / 概念 | 作用 |
|---|---|
| Binder(绑定器) | 对接具体 MQ 的适配器,如 rocketmq-binder、kafka-binder,屏蔽 MQ 差异。 |
| Binding(绑定) | 定义业务代码与 Binder 的连接关系,分为 input(消费)和 output(生产)。 |
| Destination | 消息的目标地址(对应 MQ 的 Topic/Exchange),由 Binder 映射到具体 MQ 的资源。 |
| Group | 消费组,保证消息仅被消费组内一个实例消费(集群消费)。 |
| StreamBridge | 动态发送消息的核心类,替代传统的 Source 接口,支持动态指定通道 / 主题。 |
Spring Cloud Stream 位于业务代码和具体 MQ 之间,通过 Binder 把统一的输入 / 输出通道映射到 RabbitMQ、RocketMQ 或 Kafka。
#mermaid-svg-gDN5F79rX4ukfReM{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-gDN5F79rX4ukfReM .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-gDN5F79rX4ukfReM .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-gDN5F79rX4ukfReM .error-icon{fill:#552222;}#mermaid-svg-gDN5F79rX4ukfReM .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-gDN5F79rX4ukfReM .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-gDN5F79rX4ukfReM .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-gDN5F79rX4ukfReM .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-gDN5F79rX4ukfReM .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-gDN5F79rX4ukfReM .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-gDN5F79rX4ukfReM .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-gDN5F79rX4ukfReM .marker{fill:#333333;stroke:#333333;}#mermaid-svg-gDN5F79rX4ukfReM .marker.cross{stroke:#333333;}#mermaid-svg-gDN5F79rX4ukfReM svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-gDN5F79rX4ukfReM p{margin:0;}#mermaid-svg-gDN5F79rX4ukfReM .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-gDN5F79rX4ukfReM .cluster-label text{fill:#333;}#mermaid-svg-gDN5F79rX4ukfReM .cluster-label span{color:#333;}#mermaid-svg-gDN5F79rX4ukfReM .cluster-label span p{background-color:transparent;}#mermaid-svg-gDN5F79rX4ukfReM .label text,#mermaid-svg-gDN5F79rX4ukfReM span{fill:#333;color:#333;}#mermaid-svg-gDN5F79rX4ukfReM .node rect,#mermaid-svg-gDN5F79rX4ukfReM .node circle,#mermaid-svg-gDN5F79rX4ukfReM .node ellipse,#mermaid-svg-gDN5F79rX4ukfReM .node polygon,#mermaid-svg-gDN5F79rX4ukfReM .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-gDN5F79rX4ukfReM .rough-node .label text,#mermaid-svg-gDN5F79rX4ukfReM .node .label text,#mermaid-svg-gDN5F79rX4ukfReM .image-shape .label,#mermaid-svg-gDN5F79rX4ukfReM .icon-shape .label{text-anchor:middle;}#mermaid-svg-gDN5F79rX4ukfReM .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-gDN5F79rX4ukfReM .rough-node .label,#mermaid-svg-gDN5F79rX4ukfReM .node .label,#mermaid-svg-gDN5F79rX4ukfReM .image-shape .label,#mermaid-svg-gDN5F79rX4ukfReM .icon-shape .label{text-align:center;}#mermaid-svg-gDN5F79rX4ukfReM .node.clickable{cursor:pointer;}#mermaid-svg-gDN5F79rX4ukfReM .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-gDN5F79rX4ukfReM .arrowheadPath{fill:#333333;}#mermaid-svg-gDN5F79rX4ukfReM .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-gDN5F79rX4ukfReM .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-gDN5F79rX4ukfReM .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-gDN5F79rX4ukfReM .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-gDN5F79rX4ukfReM .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-gDN5F79rX4ukfReM .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-gDN5F79rX4ukfReM .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-gDN5F79rX4ukfReM .cluster text{fill:#333;}#mermaid-svg-gDN5F79rX4ukfReM .cluster span{color:#333;}#mermaid-svg-gDN5F79rX4ukfReM div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-gDN5F79rX4ukfReM .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-gDN5F79rX4ukfReM rect.text{fill:none;stroke-width:0;}#mermaid-svg-gDN5F79rX4ukfReM .icon-shape,#mermaid-svg-gDN5F79rX4ukfReM .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-gDN5F79rX4ukfReM .icon-shape p,#mermaid-svg-gDN5F79rX4ukfReM .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-gDN5F79rX4ukfReM .icon-shape .label rect,#mermaid-svg-gDN5F79rX4ukfReM .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-gDN5F79rX4ukfReM .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-gDN5F79rX4ukfReM .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-gDN5F79rX4ukfReM :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 业务代码
Supplier / Function / Consumer / StreamBridge
Binding
input / output 通道
Binder
中间件适配层
RabbitMQ Binder
Exchange / Queue
RocketMQ Binder
Topic / Tag
Kafka Binder
Topic / Partition
RabbitMQ 集群
RocketMQ 集群
Kafka 集群
4. 使用示例(RocketMQ/Kafka/RabbitMQ)
1. 依赖引入
xml
<!-- ========== RabbitMQ Binder ========== -->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-stream-rabbit</artifactId>
</dependency>
<!-- ========== Kafka Binder ========== -->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-stream-kafka</artifactId>
<!-- 强制指定版本,避免依赖解析错误 -->
<version>3.2.8</version>
</dependency>
<!-- ========== RocketMQ Binder ========== -->
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-starter-stream-rocketmq</artifactId>
</dependency>
2. 核心配置文件(application.yml)
yaml
spring:
cloud:
stream:
default-binder: rabbit
default-content-type: application/json
# ========== RocketMQ Binder 全局配置 ==========
rocketmq:
binder:
name-server: 192.168.1.18:9876 # 强制指定NameServer
bindings:
rocketmq-output:
producer:
sync: true
tag: rocketmq-tag
rocketmq-input:
consumer:
tags: rocketmq-tag
broadcasting: false
# ========== 各绑定器配置 ==========
binders:
rabbit:
type: rabbit
environment:
spring:
rabbitmq:
host: 192.168.1.18
port: 5672
username: admin
password: 123456
virtual-host: /my_vhost
rocketmq:
type: rocketmq
environment:
spring:
rocketmq:
name-server: 192.168.1.18:9876 # 双重保障
producer:
group: stream-rocketmq-producer-group
kafka:
type: kafka
environment:
spring:
kafka:
bootstrap-servers: 192.168.1.18:9092
consumer:
group-id: stream-kafka-consumer-group
auto-offset-reset: earliest
# ========== 消息通道绑定 ==========
bindings:
# RabbitMQ通道
rabbit-output:
destination: rabbit-exchange
binder: rabbit
producer:
required-groups: rabbit-group
rabbit-input:
destination: rabbit-exchange
binder: rabbit
group: rabbit-group
# demo通道(RabbitMQ)
demo-output:
destination: demo-exchange
content-type: application/json
binder: rabbit
producer:
required-groups: demo-group
demo-input:
destination: demo-exchange
content-type: application/json
binder: rabbit
group: demo-group
# RocketMQ通道
rocketmq-output:
destination: rocketmq-topic
binder: rocketmq
producer:
required-groups: rocketmq-group
rocketmq-input:
destination: rocketmq-topic
binder: rocketmq
group: rocketmq-group
# Kafka通道
kafka-output:
destination: kafka-topic
binder: kafka
producer:
required-groups: kafka-group
kafka-input:
destination: kafka-topic
binder: kafka
group: kafka-group
# ========== 各中间件扩展配置 ==========
rabbit:
bindings:
rabbit-output:
producer:
exchangeType: topic
routingKeyExpression: '''rabbit-key'''
rabbit-input:
consumer:
bindingRoutingKey: rabbit-key
acknowledgeMode: auto
demo-output:
producer:
exchangeType: topic
routingKeyExpression: '''demo-key'''
demo-input:
consumer:
bindingRoutingKey: demo-key
acknowledgeMode: manual
kafka:
bindings:
kafka-output:
producer:
compression-type: none
kafka-input:
consumer:
enable-auto-commit: true
auto-commit-interval: 1000
3. 统一消息通道定义
java
package com.microlink.provider.config;
import org.springframework.cloud.stream.annotation.Input;
import org.springframework.cloud.stream.annotation.Output;
import org.springframework.messaging.MessageChannel;
import org.springframework.messaging.SubscribableChannel;
/**
* 统一消息通道定义:隔离不同中间件的通道
*/
public interface StreamChannels {
// ========== RabbitMQ 通道 ==========
String RABBIT_OUTPUT = "rabbit-output";
String RABBIT_INPUT = "rabbit-input";
// ========== RocketMQ 通道 ==========
String ROCKETMQ_OUTPUT = "rocketmq-output";
String ROCKETMQ_INPUT = "rocketmq-input";
// ========== Kafka 通道 ==========
String KAFKA_OUTPUT = "kafka-output";
String KAFKA_INPUT = "kafka-input";
// ------------------------------ RabbitMQ ------------------------------
@Output(RABBIT_OUTPUT)
MessageChannel rabbitOutput();
@Input(RABBIT_INPUT)
SubscribableChannel rabbitInput();
// ------------------------------ RocketMQ ------------------------------
@Output(ROCKETMQ_OUTPUT)
MessageChannel rocketmqOutput();
@Input(ROCKETMQ_INPUT)
SubscribableChannel rocketmqInput();
// ------------------------------ Kafka ------------------------------
@Output(KAFKA_OUTPUT)
MessageChannel kafkaOutput();
@Input(KAFKA_INPUT)
SubscribableChannel kafkaInput();
}
4. 通用消息生产者
java
package com.microlink.provider.service;
import com.microlink.provider.config.StreamChannels;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.messaging.support.MessageBuilder;
import org.springframework.stereotype.Service;
/**
* 通用消息生产者:支持发送到不同中间件
*/
@Slf4j
@Service
@RequiredArgsConstructor
public class MessageProducerService {
private final StreamChannels streamChannels;
/**
* 发送消息到 RabbitMQ
*/
public void sendToRabbitMQ(String message) {
boolean result = streamChannels.rabbitOutput()
.send(MessageBuilder.withPayload(message).build());
log.info("发送到 RabbitMQ {}:{}", result ? "成功" : "失败", message);
}
/**
* 发送消息到 RocketMQ
*/
public void sendToRocketMQ(String message) {
boolean result = streamChannels.rocketmqOutput()
.send(MessageBuilder.withPayload(message).build());
log.info("发送到 RocketMQ {}:{}", result ? "成功" : "失败", message);
}
/**
* 发送消息到 Kafka
*/
public void sendToKafka(String message) {
boolean result = streamChannels.kafkaOutput()
.send(MessageBuilder.withPayload(message).build());
log.info("发送到 Kafka {}:{}", result ? "成功" : "失败", message);
}
}
5. 多中间件消费者
java
package com.microlink.provider.component;
import com.microlink.provider.config.StreamChannels;
import lombok.extern.slf4j.Slf4j;
import org.springframework.cloud.stream.annotation.StreamListener;
import org.springframework.messaging.Message;
import org.springframework.stereotype.Component;
/**
* 多中间件消息消费者
*/
@Slf4j
@Component
public class MessageConsumerListener {
/**
* 消费 RabbitMQ 消息
*/
@StreamListener(StreamChannels.RABBIT_INPUT)
public void consumeRabbitMQ(Message<String> message) {
log.info("消费 RabbitMQ 消息:{}", message.getPayload());
}
/**
* 消费 RocketMQ 消息
*/
@StreamListener(StreamChannels.ROCKETMQ_INPUT)
public void consumeRocketMQ(Message<String> message) {
log.info("消费 RocketMQ 消息:{}", message.getPayload());
}
/**
* 消费 Kafka 消息
*/
@StreamListener(StreamChannels.KAFKA_INPUT)
public void consumeKafka(Message<String> message) {
log.info("消费 Kafka 消息:{}", message.getPayload());
}
}
6. 测试控制器
java
package com.microlink.provider.controller;
import com.microlink.provider.service.MessageProducerService;
import lombok.RequiredArgsConstructor;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
/**
* 测试接口:发送消息到不同中间件
*/
@RestController
@RequestMapping("/message")
@RequiredArgsConstructor
public class MessageController {
private final MessageProducerService producerService;
/**
* 发送到 RabbitMQ
*/
@GetMapping("/rabbit/{msg}")
public String sendToRabbit(@PathVariable String msg) {
producerService.sendToRabbitMQ(msg);
return "已发送到 RabbitMQ:" + msg;
}
/**
* 发送到 RocketMQ
*/
@GetMapping("/rocketmq/{msg}")
public String sendToRocketMQ(@PathVariable String msg) {
producerService.sendToRocketMQ(msg);
return "已发送到 RocketMQ:" + msg;
}
/**
* 发送到 Kafka
*/
@GetMapping("/kafka/{msg}")
public String sendToKafka(@PathVariable String msg) {
producerService.sendToKafka(msg);
return "已发送到 Kafka:" + msg;
}
}
7. 项目启动类添加注解
java
@EnableBinding(StreamChannels.class)
通过观察可以明显看出:
- 使用 Spring Cloud Stream:切换 MQ 仅需修改application.yml配置,业务代码完全不变;
- 不使用 Stream:切换 MQ 需要重写所有生产者 / 消费者代码,耦合度极高。
5. Spring Cloud Stream 的核心优势
- 解耦 MQ 依赖:业务代码面向 Stream 的抽象 API 编程,而非具体 MQ 的 SDK,降低技术绑定;
- 简化配置与开发:自动管理 MQ 客户端的生命周期(无需手动 start()/shutdown()),内置重试、批量消费、消息确认等能力;
- 统一的消息语义:统一的消费组、分区、重试、死信等语义,无需学习不同 MQ 的差异化概念;
- 微服务适配友好:与 Spring Cloud 生态深度集成(如与 Spring Cloud Bus、Sleuth 链路追踪整合);
- 动态消息发送:StreamBridge 支持动态指定主题 / 通道,解决传统静态绑定的局限性。
6. 什么时候可以不用 Spring Cloud Stream?
虽然 Stream 优势显著,但以下场景可以考虑原生 SDK:
- 极简场景:单项目仅使用一种 MQ,且无需切换,对代码耦合度不敏感;
- 极致性能要求:Stream 的抽象层会带来极轻微的性能损耗(几乎可忽略),超高性能场景可考虑原生 SDK;
- 使用 MQ 的高级特性:部分 MQ 的极致个性化特性(如 Kafka 的流处理、RabbitMQ 的自定义交换机),Stream 的抽象层可能未完全封装,需用原生 API;
- 非 Spring 生态项目:如纯 JavaSE 项目、非 Spring Boot 项目,使用 Stream 反而增加依赖。
五、三者对比
1. 核心维度对比表
| 维度 | RocketMQ | Kafka | RabbitMQ |
|---|---|---|---|
| 核心优势 | 事务消息、延迟消息、阿里生态适配 | 超高吞吐量、大数据流处理 | 路由灵活、协议丰富、易用性高 |
| 吞吐量 | 高(10 万 +/ 秒) | 极高(百万+/ 秒) | 中(万级/ 秒) |
| 可靠性 | 高(支持事务、重试、死信) | 中(依赖副本,弱事务) | 高(支持确认、死信) |
| 延迟消息 | 原生支持 | 需自定义开发 | 需安装插件 |
| 事务消息 | 原生支持 | 不支持 | 需自定义 |
| 路由灵活性 | 中(主题/ 标签) | 低(仅分区) | 极高(多种交换机) |
| Spring Cloud 适配 | 最佳(Alibaba 原生) | 良好(Stream 原生) | 良好(Stream 原生) |
2. 场景选型
#mermaid-svg-o6pl83KVGl0yElgu{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-o6pl83KVGl0yElgu .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-o6pl83KVGl0yElgu .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-o6pl83KVGl0yElgu .error-icon{fill:#552222;}#mermaid-svg-o6pl83KVGl0yElgu .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-o6pl83KVGl0yElgu .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-o6pl83KVGl0yElgu .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-o6pl83KVGl0yElgu .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-o6pl83KVGl0yElgu .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-o6pl83KVGl0yElgu .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-o6pl83KVGl0yElgu .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-o6pl83KVGl0yElgu .marker{fill:#333333;stroke:#333333;}#mermaid-svg-o6pl83KVGl0yElgu .marker.cross{stroke:#333333;}#mermaid-svg-o6pl83KVGl0yElgu svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-o6pl83KVGl0yElgu p{margin:0;}#mermaid-svg-o6pl83KVGl0yElgu .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-o6pl83KVGl0yElgu .cluster-label text{fill:#333;}#mermaid-svg-o6pl83KVGl0yElgu .cluster-label span{color:#333;}#mermaid-svg-o6pl83KVGl0yElgu .cluster-label span p{background-color:transparent;}#mermaid-svg-o6pl83KVGl0yElgu .label text,#mermaid-svg-o6pl83KVGl0yElgu span{fill:#333;color:#333;}#mermaid-svg-o6pl83KVGl0yElgu .node rect,#mermaid-svg-o6pl83KVGl0yElgu .node circle,#mermaid-svg-o6pl83KVGl0yElgu .node ellipse,#mermaid-svg-o6pl83KVGl0yElgu .node polygon,#mermaid-svg-o6pl83KVGl0yElgu .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-o6pl83KVGl0yElgu .rough-node .label text,#mermaid-svg-o6pl83KVGl0yElgu .node .label text,#mermaid-svg-o6pl83KVGl0yElgu .image-shape .label,#mermaid-svg-o6pl83KVGl0yElgu .icon-shape .label{text-anchor:middle;}#mermaid-svg-o6pl83KVGl0yElgu .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-o6pl83KVGl0yElgu .rough-node .label,#mermaid-svg-o6pl83KVGl0yElgu .node .label,#mermaid-svg-o6pl83KVGl0yElgu .image-shape .label,#mermaid-svg-o6pl83KVGl0yElgu .icon-shape .label{text-align:center;}#mermaid-svg-o6pl83KVGl0yElgu .node.clickable{cursor:pointer;}#mermaid-svg-o6pl83KVGl0yElgu .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-o6pl83KVGl0yElgu .arrowheadPath{fill:#333333;}#mermaid-svg-o6pl83KVGl0yElgu .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-o6pl83KVGl0yElgu .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-o6pl83KVGl0yElgu .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-o6pl83KVGl0yElgu .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-o6pl83KVGl0yElgu .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-o6pl83KVGl0yElgu .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-o6pl83KVGl0yElgu .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-o6pl83KVGl0yElgu .cluster text{fill:#333;}#mermaid-svg-o6pl83KVGl0yElgu .cluster span{color:#333;}#mermaid-svg-o6pl83KVGl0yElgu div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-o6pl83KVGl0yElgu .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-o6pl83KVGl0yElgu rect.text{fill:none;stroke-width:0;}#mermaid-svg-o6pl83KVGl0yElgu .icon-shape,#mermaid-svg-o6pl83KVGl0yElgu .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-o6pl83KVGl0yElgu .icon-shape p,#mermaid-svg-o6pl83KVGl0yElgu .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-o6pl83KVGl0yElgu .icon-shape .label rect,#mermaid-svg-o6pl83KVGl0yElgu .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-o6pl83KVGl0yElgu .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-o6pl83KVGl0yElgu .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-o6pl83KVGl0yElgu :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 是
否
是
否
是
否
高吞吐
企业业务可靠性
中小型通用场景
开始选型
是否强依赖事务消息
或延迟消息?
优先选择 RocketMQ
是否以日志采集、埋点
大数据流处理为主?
优先选择 Kafka
是否需要复杂路由
协议互通或灵活交换机?
优先选择 RabbitMQ
系统规模与吞吐要求是否很高?
| 场景类型 | 推荐 MQ | 核心原因 |
|---|---|---|
| Spring Cloud Alibaba 微服务 | RocketMQ | 生态适配最佳,支持事务/ 延迟消息 |
| 日志收集/ 大数据分析 | Kafka | 超高吞吐量,适合海量数据传输 |
| 复杂路由/ 跨系统对接 | RabbitMQ | 路由规则丰富,AMQP 协议通用性强 |
| 分布式事务保障 | RocketMQ | 唯一原生支持事务消息的 MQ |
| 中小型系统快速开发 | RabbitMQ | 部署简单,易用性高 |
六、写在最后
- RocketMQ 是 Spring Cloud Alibaba 微服务的首选,尤其适合需要事务、延迟消息的电商 / 金融场景;
- Kafka 聚焦高吞吐的大数据场景,如日志、实时计算,不适合强事务要求的业务;
- RabbitMQ 胜在路由灵活性,适合中小型系统或复杂路由场景,吞吐量是其短板;
- 基于 Spring Cloud Stream 可以用统一的 API 适配三款 MQ,切换仅需修改配置,降低了技术绑定风险;
- Spring Cloud Stream 不是整合 MQ 的必需品,但却是 Spring Cloud Alibaba 微服务架构中的优选方案。