项目场景:
Flink使用cdc读取mysql配置信息
问题描述
print没有输出
java
public static MySqlSource<String> getMySqlCDCSource() {
Map<String, Object> config = new HashMap<>();
config.put(JsonConverterConfig.DECIMAL_FORMAT_CONFIG, DecimalFormat.NUMERIC.name());
JsonDebeziumDeserializationSchema jdd = new JsonDebeziumDeserializationSchema(false, config);
return MySqlSource.<String>builder()
.hostname(readValue("mysql.host"))
.port(Integer.parseInt(readValue("mysql.port")))
.databaseList(readValue("mysql.database"))//库
.tableList(readValue("mysql.table"))//表
.username(readValue("mysql.username"))
.password(readValue("mysql.password"))
.serverTimeZone(readValue("mysql.serverTimeZone"))
.startupOptions(StartupOptions.initial())//启动的时候从第一次开始读取
.deserializer(jdd)
.build();
}
public static void main(String[] args) throws Exception {
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
SingleOutputStreamOperator<String> cdcConfig = env.fromSource(MysqlUtil.getMySqlCDCSource(), WatermarkStrategy.noWatermarks(), "mysqlConfig")
.uid("mysqlSource_sourceName_uid").name("mysqlSource_sourceName_name").setParallelism(1);
cdcConfig.print();
env.execute();
}
原因分析:
由于Sink是Mysql,导入了mysql jdbc依赖
xml
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>8.0.30</version>
</dependency>
查阅资料后,可能是MySQL Connector/J 的版本不兼容引起的。MySQL Connector/J 8.0.23 与 Flink 的 CDC 版本兼容性较好,但其他版本可能存在不兼容的情况。尝试将版本改为8.0.23后问题解决,输出正常
解决方案:
将mysql jdbc依赖版本改为8.0.23
xml
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>8.0.23</version>
</dependency>