使用 Canal实时监听数据库变化

业务场景

我用这个东西的需求很简单当你需要监听数据库表中某个值的变化,且对实时性要求很高的时候,比如这个订单变化一变, 你就需要做一些操作, 诶这个时候你就可以使用 Canal, 当然你如果对这个实时性要求的不那么高, 你这个订单状态变化之后, 你拿个定时器扫一遍, 在集中做处理也可以, 不同的业务场景有不同的解决办法

安装这个之前你得把数据库的binlog日志给打开

mysql的环境部署

把这个my.cnf这个配置文件中配置

bash 复制代码

[mysqld]
# 打开binlog
log-bin=mysql-bin
# 选择ROW(行)模式
binlog-format=ROW
# 配置MySQL replaction需要定义，不要和canal的slaveId重复
server_id=1

然后重启你的mysql

执行这个sql 然后看一下你的配置有没有起效果

bash 复制代码

show variables like 'log_bin'

在看一下你目前mysql的偏移量

bash 复制代码

show master status

在执行以下sql

bash 复制代码

-- 创建用户 用户名：canal 密码：canal
create user 'canal'@'%' identified by 'canal';
-- 授权 *.*表示所有库
grant SELECT, REPLICATION SLAVE, REPLICATION CLIENT on *.* to 'canal'@'%' identified by 'canal';

下载和安装

https://github.com/alibaba/canal/releases?page=1

解压出来

接着打开配置文件conf/example/instance.properties，配置信息如下：

bash 复制代码

# 数据库地址
canal.instance.master.address=127.0.0.1:3306
# binlog日志名称  canal.instance.master.journal.name 是 show master status 中的file字段
canal.instance.master.journal.name=mysql-bin.000001
# mysql主库链接时起始的binlog偏移量 anal.instance.master.journal.name 是 show master status 中的position字段
canal.instance.master.position=154

# username/password
# 在MySQL服务器授权的账号密码
canal.instance.dbUsername=canal
canal.instance.dbPassword=canal
# 监听所有的表
canal.instance.filter.regex=.*\\..*

然后执行一下

startup.bat

踩坑了哈

然后我就发现报错了

bash 复制代码

2025-12-08 11:52:55.620 [destination = example , address = /127.0.0.1:3306 , EventParser] ERROR c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - dump address /127.0.0.1:3306 has an error, retrying. caused by java.lang.NullPointerException: null

结果发现了我的配置文件写错了我把server_id 写成了server-id

粗心了哈, 大家在配置的时候还是要多多检查

再次启动依旧报错

bash 复制代码

2025-12-08 12:01:11.499 [destination = example , address = localhost/127.0.0.1:3306 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - ---> begin to find start position, it will be long time for reset or first position 2025-12-08 12:01:11.504 [destination = example , address = localhost/127.0.0.1:3306 , EventParser] WARN c.a.otter.canal.parse.inbound.mysql.MysqlConnection - load MySQL @@version_comment : MySQL Community Server - GPL 2025-12-08 12:01:11.504 [destination = example , address = localhost/127.0.0.1:3306 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - prepare to find start position LAPTOP-8BOGI302-bin.000153:4:1765158688000 2025-12-08 12:01:11.817 [destination = example , address = localhost/127.0.0.1:3306 , EventParser] ERROR c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - dump address localhost/127.0.0.1:3306 has an error, retrying. caused by java.lang.NullPointerException: null at com.alibaba.polardbx.druid.sql.visitor.SQLASTOutputVisitor.visit(SQLASTOutputVisitor.java:6029) at com.alibaba.polardbx.druid.sql.ast.statement.SQLCheck.accept0(SQLCheck.java:66) at com.alibaba.polardbx.druid.sql.ast.SQLObjectImpl.accept(SQLObjectImpl.java:47) at com.alibaba.polardbx.druid.sql.visitor.SQLASTOutputVisitor.printTableElements(SQLASTOutputVisitor.java:3875) at com.alibaba.polardbx.druid.sql.visitor.SQLASTOutputVisitor.visit(SQLASTOutputVisitor.java:10663) at com.alibaba.polardbx.druid.sql.dialect.mysql.ast.statement.MySqlCreateTableStatement.accept0(MySqlCreateTableStatement.java:192) at com.alibaba.polardbx.druid.sql.dialect.mysql.ast.statement.MySqlCreateTableStatement.accept0(MySqlCreateTableStatement.java:185) at com.alibaba.polardbx.druid.sql.ast.SQLObjectImpl.accept(SQLObjectImpl.java:47) at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.MemoryTableMeta.snapshot(MemoryTableMeta.java:159) at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta.applySnapshotToDB(DatabaseTableMeta.java:325) at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta.rollback(DatabaseTableMeta.java:176) at com.alibaba.otter.canal.parse.inbound.mysql.AbstractMysqlEventParser.processTableMeta(AbstractMysqlEventParser.java:144) at com.alibaba.otter.canal.parse.inbound.AbstractEventParser$1.run(AbstractEventParser.java:192) at java.lang.Thread.run(Unknown Source) 2025-12-08 12:01:11.817 [destination = example , address = localhost/127.0.0.1:3306 , EventParser] ERROR com.alibaba.otter.canal.common.alarm.LogAlarmHandler - destination:example[java.lang.NullPointerException at com.alibaba.polardbx.druid.sql.visitor.SQLASTOutputVisitor.visit(SQLASTOutputVisitor.java:6029) at com.alibaba.polardbx.druid.sql.ast.statement.SQLCheck.accept0(SQLCheck.java:66) at com.alibaba.polardbx.druid.sql.ast.SQLObjectImpl.accept(SQLObjectImpl.java:47) at com.alibaba.polardbx.druid.sql.visitor.SQLASTOutputVisitor.printTableElements(SQLASTOutputVisitor.java:3875) at com.alibaba.polardbx.druid.sql.visitor.SQLASTOutputVisitor.visit(SQLASTOutputVisitor.java:10663) at com.alibaba.polardbx.druid.sql.dialect.mysql.ast.statement.MySqlCreateTableStatement.accept0(MySqlCreateTableStatement.java:192) at com.alibaba.polardbx.druid.sql.dialect.mysql.ast.statement.MySqlCreateTableStatement.accept0(MySqlCreateTableStatement.java:185) at com.alibaba.polardbx.druid.sql.ast.SQLObjectImpl.accept(SQLObjectImpl.java:47) at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.MemoryTableMeta.snapshot(MemoryTableMeta.java:159) at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta.applySnapshotToDB(DatabaseTableMeta.java:325) at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta.rollback(DatabaseTableMeta.java:176) at com.alibaba.otter.canal.parse.inbound.mysql.AbstractMysqlEventParser.processTableMeta(AbstractMysqlEventParser.java:144) at com.alibaba.otter.canal.parse.inbound.AbstractEventParser$1.run(AbstractEventParser.java:192) at java.lang.Thread.run(Unknown Source)

问了GPT发现

GPT说是:MySQL 8.0+ 自动创建一些 CHECK 约束，Canal 无法解析，会导致 MemoryTableMeta snapshot 阶段崩溃。

所以只要在instance.properties中把

canal.instance.tsdb.enable = true

改为

canal.instance.tsdb.enable = false

启动完毕,成功运行

java代码, 我这里是SpringBoot

java 复制代码

@Slf4j
@Component
public class CannalClient implements InitializingBean {

@Override
    public void afterPropertiesSet() throws Exception {
        Thread canalThread = new Thread(this::startCanalListener, "canal-listener");
        canalThread.setDaemon(true); // 设置为守护线程，不要影响 SpringBoot 启动
        canalThread.start();
    }
	private void startCanalListener() {
        CanalConnector connector = CanalConnectors
                .newSingleConnector(new InetSocketAddress("127.0.0.1", 11111), "example", "", "");
				
				try {
            connector.connect();
			// TODO 这里按你自己的数据库名和要监控的表, 尽量不要全部监控
            connector.subscribe("数据库名\\.表名1|数据库名\\.表名1");
            connector.rollback();
            while (true) {
                Message message = connector.getWithoutAck(100);
                long batchId = message.getId();
                if (batchId == -1 || message.getEntries().isEmpty()) {
                    Thread.sleep(500); // 不会卡住 Spring 主线程
                    continue;
                }
                for (CanalEntry.Entry entry : message.getEntries()) {
                    if (entry.getEntryType() != CanalEntry.EntryType.ROWDATA) continue;
                    CanalEntry.RowChange rowChange = CanalEntry.RowChange.parseFrom(entry.getStoreValue());
                    if (rowChange.getEventType() != CanalEntry.EventType.UPDATE) continue;

                    String tableName = entry.getHeader().getTableName();
                    rowChange.getRowDatasList().forEach(rowData ->
                            logRowUpdate(tableName, rowData)
                    );
                }
                connector.ack(batchId);
            }
        } catch (Exception e) {
            log.error("Canal监听出错: ", e);
        } finally {
            connector.disconnect();
        }
				
				}
				
				
		private void logRowUpdate(String tableName, CanalEntry.RowData rowData) {
        log.info("===== 表：{} 更新 =====", tableName);
        List<CanalEntry.Column> beforeColumns = rowData.getBeforeColumnsList();
        List<CanalEntry.Column> afterColumns = rowData.getAfterColumnsList();
        for (int i = 0; i < beforeColumns.size(); i++) {
            String primaryKey = null;
            CanalEntry.Column beforeCol = beforeColumns.get(i);
            CanalEntry.Column afterCol = afterColumns.get(i);
            // 先找主键字段
            if (beforeCol.getIsKey()) {
                primaryKey = beforeCol.getValue();
            }
            if (!beforeCol.getValue().equals(afterCol.getValue())) {
                log.info("字段变更：{}", beforeCol.getName());
                log.info("   BEFORE: {}", beforeCol.getValue());
                log.info("   AFTER : {}", afterCol.getValue());
                // TODO 这里你做你想做的
            }
        }
    }
				
}