RocketMQ可视化监控与管理

RocketMQ可视化监控与管理

RocketMQ Dashboard安装与配置

1. 下载和部署

# 下载RocketMQ Dashboard
wget https://archive.apache.org/dist/rocketmq/rocketmq-dashboard/1.0.0/rocketmq-dashboard-1.0.0.jar

# 启动Dashboard
java -jar rocketmq-dashboard-1.0.0.jar --server.port=8080 --rocketmq.config.namesrvAddr=localhost:9876

2. Docker部署方式

# 使用Docker运行RocketMQ Dashboard
docker run -d --name rocketmq-dashboard \
  -p 8080:8080 \
  -e "JAVA_OPTS=-Drocketmq.namesrv.addr=localhost:9876" \
  apacherocketmq/rocketmq-dashboard:1.0.0

3. 配置文件详解

# application.properties
# NameServer地址
rocketmq.config.namesrvAddr=localhost:9876

# 是否启用权限控制
rocketmq.config.isVIPChannel=false

# 数据刷新间隔（毫秒）
rocketmq.config.dataRefreshInterval=5000

# 是否启用消息轨迹
rocketmq.config.enableMsgTrace=true

# 消息轨迹Topic
rocketmq.config.msgTraceTopic=RMQ_SYS_TRACE_TOPIC

# 访问控制
rocketmq.config.accessKey=
rocketmq.config.secretKey=

Dashboard功能详解

1. 集群概览

Dashboard首页展示集群的整体状态信息：

• Broker节点状态
• Topic数量统计
• Consumer Group数量统计
• 消息生产TPS
• 消息消费TPS

2. Topic管理

Topic列表

• 查看所有Topic信息
• Topic消息总量统计
• Topic队列分布情况
• Topic创建和删除操作

Topic详情

// 通过代码创建Topic
public class TopicManager {
    public void createTopic() throws Exception {
        DefaultMQAdminExt adminExt = new DefaultMQAdminExt();
        adminExt.setNamesrvAddr("localhost:9876");
        adminExt.start();
        
        TopicConfig topicConfig = new TopicConfig();
        topicConfig.setTopicName("NewTopic");
        topicConfig.setReadQueueNums(8);
        topicConfig.setWriteQueueNums(8);
        topicConfig.setPerm(6); // 读写权限
        
        adminExt.createAndUpdateTopicConfig("192.168.1.10:10911", topicConfig);
        adminExt.shutdown();
    }
}

3. Consumer Group管理

Consumer Group列表

• 查看所有Consumer Group状态
• Consumer Group订阅关系
• 消费进度监控
• 积压消息数量

Consumer Group详情

• 查看消费者实例列表
• 每个队列的消费位点
• 消费延迟统计
• 消费TPS监控

4. 消息查询

按消息ID查询

// 通过Admin工具查询消息
public class MessageQuery {
    public void queryByMessageId() throws Exception {
        DefaultMQAdminExt adminExt = new DefaultMQAdminExt();
        adminExt.setNamesrvAddr("localhost:9876");
        adminExt.start();
        
        MessageExt msg = adminExt.viewMessage("TopicTest", "MessageId");
        System.out.println("Message Body: " + new String(msg.getBody()));
        System.out.println("Message Properties: " + msg.getProperties());
        
        adminExt.shutdown();
    }
}

按消息Key查询

// 按Key查询消息
public void queryByKey() throws Exception {
    DefaultMQAdminExt adminExt = new DefaultMQAdminExt();
    adminExt.setNamesrvAddr("localhost:9876");
    adminExt.start();
    
    QueryResult result = adminExt.queryMessage("TopicTest", "KEY123", 32, 0, System.currentTimeMillis());
    for (MessageExt msg : result.getMessageList()) {
        System.out.println("Found message: " + new String(msg.getBody()));
    }
    
    adminExt.shutdown();
}

自定义监控指标

1. 生产者监控

public class ProducerMonitor {
    private DefaultMQProducer producer;
    private MeterRegistry meterRegistry;
    
    public ProducerMonitor(MeterRegistry meterRegistry) {
        this.meterRegistry = meterRegistry;
        this.producer = new DefaultMQProducer("monitor_producer_group");
        
        // 注册监控指标
        Gauge.builder("rocketmq.producer.send.tps")
            .register(meterRegistry, this, value -> getSendTps());
            
        Gauge.builder("rocketmq.producer.send.latency")
            .register(meterRegistry, this, value -> getAverageLatency());
    }
    
    private double getSendTps() {
        // 实现TPS计算逻辑
        return 0.0;
    }
    
    private double getAverageLatency() {
        // 实现延迟计算逻辑
        return 0.0;
    }
}

2. 消费者监控

public class ConsumerMonitor {
    private DefaultMQPushConsumer consumer;
    private MeterRegistry meterRegistry;
    
    public ConsumerMonitor(MeterRegistry meterRegistry) {
        this.meterRegistry = meterRegistry;
        this.consumer = new DefaultMQPushConsumer("monitor_consumer_group");
        
        // 注册监控指标
        Gauge.builder("rocketmq.consumer.consume.tps")
            .register(meterRegistry, this, value -> getConsumeTps());
            
        Gauge.builder("rocketmq.consumer.backlog")
            .register(meterRegistry, this, value -> getBacklogCount());
    }
    
    private double getConsumeTps() {
        // 实现消费TPS计算逻辑
        return 0.0;
    }
    
    private long getBacklogCount() {
        // 实现积压消息数量计算逻辑
        return 0L;
    }
}

告警策略配置

1. Prometheus告警规则

# prometheus-rules.yml
groups:
- name: rocketmq.rules
  rules:
  - alert: RocketMQHighBacklog
    expr: rocketmq_consumer_backlog > 100000
    for: 5m
    labels:
      severity: warning
    annotations:
      summary: "RocketMQ consumer backlog is high"
      description: "Consumer group {{ $labels.consumer_group }} has {{ $value }} messages backlog"
      
  - alert: RocketMQHighLatency
    expr: rocketmq_producer_send_latency > 1000
    for: 2m
    labels:
      severity: critical
    annotations:
      summary: "RocketMQ producer latency is high"
      description: "Producer send latency is {{ $value }}ms"
      
  - alert: RocketMQLowConsumerCount
    expr: rocketmq_consumer_count < 1
    for: 1m
    labels:
      severity: critical
    annotations:
      summary: "RocketMQ consumer count is low"
      description: "Consumer group {{ $labels.consumer_group }} has {{ $value }} active consumers"

2. 告警通知配置

# alertmanager.yml
route:
  group_by: ['alertname']
  group_wait: 10s
  group_interval: 10s
  repeat_interval: 1h
  receiver: 'webhook'

receivers:
- name: 'webhook'
  webhook_configs:
  - url: 'http://your-alert-handler.com/webhook'
    send_resolved: true

性能分析工具

1. RocketMQ Benchmarks

# 启动生产者性能测试
sh bin/tools.sh org.apache.rocketmq.example.benchmark.Producer

# 启动消费者性能测试
sh bin/tools.sh org.apache.rocketmq.example.benchmark.Consumer

# 启动事务消息测试
sh bin/tools.sh org.apache.rocketmq.example.benchmark.TransactionProducer

2. 自定义压测工具

public class CustomBenchmark {
    public static void main(String[] args) throws Exception {
        // 生产者压测
        benchmarkProducer();
        
        // 消费者压测
        benchmarkConsumer();
    }
    
    private static void benchmarkProducer() throws Exception {
        DefaultMQProducer producer = new DefaultMQProducer("benchmark_producer_group");
        producer.setNamesrvAddr("localhost:9876");
        producer.start();
        
        long startTime = System.currentTimeMillis();
        int messageCount = 100000;
        
        for (int i = 0; i < messageCount; i++) {
            Message msg = new Message("BenchmarkTopic", 
                                    ("Benchmark Message " + i).getBytes());
            producer.send(msg);
        }
        
        long endTime = System.currentTimeMillis();
        double tps = messageCount / ((endTime - startTime) / 1000.0);
        
        System.out.printf("Sent %d messages in %d ms, TPS: %.2f%n", 
                         messageCount, endTime - startTime, tps);
        
        producer.shutdown();
    }
    
    private static void benchmarkConsumer() throws Exception {
        // 实现消费者压测逻辑
    }
}

日志分析与诊断

1. 日志收集配置

# logback.xml
<configuration>
    <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>logs/rocketmq.log</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>logs/rocketmq.%d{yyyy-MM-dd}.%i.log</fileNamePattern>
            <timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
                <maxFileSize>100MB</maxFileSize>
            </timeBasedFileNamingAndTriggeringPolicy>
            <maxHistory>30</maxHistory>
        </rollingPolicy>
        <encoder>
            <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>
    
    <root level="INFO">
        <appender-ref ref="FILE" />
    </root>
</configuration>

2. 常见问题诊断

Broker性能问题

# 检查Broker GC日志
tail -f ~/logs/rocketmqlogs/gc.log

# 检查Broker内存使用情况
jstat -gc [broker_pid]

# 检查Broker线程状态
jstack [broker_pid]

网络连接问题

# 检查网络连接数
netstat -an | grep 10911 | wc -l

# 检查连接状态
netstat -an | grep 10911 | awk '{print $6}' | sort | uniq -c

小结

本章介绍了RocketMQ的可视化监控和管理工具，包括RocketMQ Dashboard的安装配置、功能使用、自定义监控指标、告警策略配置以及性能分析工具。

通过可视化的监控界面和完善的监控体系，可以及时发现和解决RocketMQ运行中的问题，确保系统的稳定性和可靠性。在下一章节中，我们将学习RocketMQ在微服务架构中的应用实践。