【flink】Chunk splitting has encountered exception

醇氧2023-08-11 14:44

执行任务报错： Chunk splitting has encountered exception

错误信息截图：

完整的错误信息：

java 复制代码 
16:30:43,911 ERROR org.apache.flink.runtime.source.coordinator.SourceCoordinator [SourceCoordinator-Source: CDC Sourceorg.jobslink.flink.UserCDCJob]  - Uncaught exception in the SplitEnumerator for Source Source: CDC Sourceorg.flink.UserCDCJob while handling operator event RequestSplitEvent (host='') from subtask 0. Triggering job failover.
org.apache.flink.util.FlinkRuntimeException: Chunk splitting has encountered exception
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.checkSplitterErrors(MySqlSnapshotSplitAssigner.java:522)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:278)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:298)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlHybridSplitAssigner.getNext(MySqlHybridSplitAssigner.java:123)
	at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.assignSplits(MySqlSourceEnumerator.java:201)
	at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.handleSplitRequest(MySqlSourceEnumerator.java:117)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$handleEventFromOperator$2(SourceCoordinator.java:230)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$9(SourceCoordinator.java:406)
	at org.apache.flink.util.ThrowableCatchingRunnable.run(ThrowableCatchingRunnable.java:40)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Error when splitting chunks for jobslink_user.jobslink_account
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitTable(MySqlSnapshotSplitAssigner.java:246)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitChunksForRemainingTables(MySqlSnapshotSplitAssigner.java:505)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	... 3 more
Caused by: java.lang.RuntimeException: Fail to analyze table in chunk splitter.
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.analyzeTable(MySqlChunkSplitter.java:154)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.splitChunks(MySqlChunkSplitter.java:117)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitTable(MySqlSnapshotSplitAssigner.java:244)
	... 7 more
Caused by: org.apache.flink.table.api.ValidationException: Incremental snapshot for tables requires primary key, but table user.account doesn't have primary key.
	at com.ververica.cdc.connectors.mysql.source.utils.ChunkUtils.getChunkKeyColumn(ChunkUtils.java:55)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.analyzeTable(MySqlChunkSplitter.java:148)
	... 9 more
16:30:43,929 INFO  org.apache.flink.runtime.jobmaster.JobMaster                 [flink-akka.actor.default-dispatcher-9]  - Trying to recover from a global failure.
org.apache.flink.util.FlinkException: Global failure triggered by OperatorCoordinator for 'Source: CDC Sourceorg.jobslink.flink.UserCDCJob -> Filter -> Flat Map -> Filter -> (Sink: Print to Std. Out, Sink: sink jobslink_account)' (operator cbc357ccb763df2852fee8c4fc7d55f2).
	at org.apache.flink.runtime.operators.coordination.OperatorCoordinatorHolder$LazyInitializedCoordinatorContext.failJob(OperatorCoordinatorHolder.java:556)
	at org.apache.flink.runtime.operators.coordination.RecreateOnResetOperatorCoordinator$QuiesceableContext.failJob(RecreateOnResetOperatorCoordinator.java:231)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinatorContext.failJob(SourceCoordinatorContext.java:316)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$9(SourceCoordinator.java:420)
	at org.apache.flink.util.ThrowableCatchingRunnable.run(ThrowableCatchingRunnable.java:40)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flink.util.FlinkRuntimeException: Chunk splitting has encountered exception
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.checkSplitterErrors(MySqlSnapshotSplitAssigner.java:522)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:278)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:298)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlHybridSplitAssigner.getNext(MySqlHybridSplitAssigner.java:123)
	at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.assignSplits(MySqlSourceEnumerator.java:201)
	at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.handleSplitRequest(MySqlSourceEnumerator.java:117)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$handleEventFromOperator$2(SourceCoordinator.java:230)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$9(SourceCoordinator.java:406)
	... 9 more
Caused by: java.lang.IllegalStateException: Error when splitting chunks for jobslink_user.jobslink_account
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitTable(MySqlSnapshotSplitAssigner.java:246)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitChunksForRemainingTables(MySqlSnapshotSplitAssigner.java:505)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	... 3 more
Caused by: java.lang.RuntimeException: Fail to analyze table in chunk splitter.
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.analyzeTable(MySqlChunkSplitter.java:154)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.splitChunks(MySqlChunkSplitter.java:117)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitTable(MySqlSnapshotSplitAssigner.java:244)
	... 7 more
Caused by: org.apache.flink.table.api.ValidationException: Incremental snapshot for tables requires primary key, but table user.account doesn't have primary key.
	at com.ververica.cdc.connectors.mysql.source.utils.ChunkUtils.getChunkKeyColumn(ChunkUtils.java:55)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.analyzeTable(MySqlChunkSplitter.java:148)
	... 9 more
16:30:43,933 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph       [flink-akka.actor.default-dispatcher-9]  - Job org.flink.UserCDCJob (35780b6f0490a368fc256b228e99f7cb) switched from state RUNNING to RESTARTING.
16:30:43,936 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph       [flink-akka.actor.default-dispatcher-9]  - Source: CDC Sourceorg.flink.UserCDCJob -> Filter -> Flat Map -> Filter -> (Sink: Print to Std. Out, Sink: sink account) (1/1) (45028a2ced14e3bbe2d7cf3349defba0) switched from RUNNING to CANCELING.

Incremental snapshot for tables requires primary key, but table user.account doesn't have primary key.

表的增量快照需要主键，但表user.account没有主键。

去user库account表增加主键

添加上主键以后就可以正常执行了！！！

相关推荐
yuanyuan3347 小时前
2026枣庄带检测报告伸缩门品牌测评｜本地服务
大数据·数据库·人工智能·伸缩门
玖米GEO运营王总监8 小时前
2026年账号矩阵协同效应怎么量化：三个指标与公式结构
java·大数据·数据库
飞飞传输21 小时前
飞驰云联文件交换建设方法论：从单点传输到持续治理
大数据·运维·安全
得帆低代码PaaS21 小时前
企业级 Agent 架构：MCP、Integration、数据与治理能力
大数据·人工智能·架构·ai agent·企业数字化·mcp
TDengine （老段）1 天前
TDengine IDMP 十大实用功能（9）
大数据·数据库·人工智能·物联网·时序数据库·tdengine·idmp
播播资源1 天前
同账户实测 11 个多模态模型：图像 0.03 元起、竖屏视频 8 元一条，最贵和最便宜差 267 倍
大数据·人工智能·深度学习
全栈陈序员1 天前
Git】从入门到精通（八）—— git log：查看提交历史
大数据·git·elasticsearch
演艺大赏1 天前
NiFi 从 3 台扩到 25 台：三轮扩容后我才明白，证书必须全量重签、安装包绝对不能拷
大数据
做科研的周师兄1 天前
【R语言实战】MixSIAR 氢氧同位素水源解析全流程：从混合空间校验到贝叶斯 MCMC 收敛诊断（后附完整代码）
java·大数据·开发语言
isfox1 天前
MapReduce 数据清洗：脏数据进来，干净数据出去，连 Reduce 都省了
大数据
热门推荐
01如何新建文件夹？ 电脑新建文件夹的4种方法02国内可直接用、免费额度/永久免费的大模型API清单（含 SiliconFlow、火山、阿里、智谱、百度、Kimi、DeepSeek、DMXAPI 等）03历年考研数学一、数学二、数学三真题试卷及答案PDF04几个好用的ip纯净度检测网站05GitHub 镜像站点06苹果iPA游戏软件资源下载网站07蜘蛛磁力 搜索引擎大全，如何使用蜘蛛磁力查找磁力链接08昇腾910B系列芯片完全指南：四款型号对比、服务器选型与大模型部署实战09微信历史版本含下载地址（ Windows PC | 安卓 | MAC ）及设置微信不更新10Dify：一站式AI应用开发平台全解析与Windows部署指南