【flink】Chunk splitting has encountered exception

醇氧2023-08-11 14:44

执行任务报错： Chunk splitting has encountered exception

错误信息截图：

完整的错误信息：

java 复制代码 
16:30:43,911 ERROR org.apache.flink.runtime.source.coordinator.SourceCoordinator [SourceCoordinator-Source: CDC Sourceorg.jobslink.flink.UserCDCJob]  - Uncaught exception in the SplitEnumerator for Source Source: CDC Sourceorg.flink.UserCDCJob while handling operator event RequestSplitEvent (host='') from subtask 0. Triggering job failover.
org.apache.flink.util.FlinkRuntimeException: Chunk splitting has encountered exception
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.checkSplitterErrors(MySqlSnapshotSplitAssigner.java:522)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:278)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:298)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlHybridSplitAssigner.getNext(MySqlHybridSplitAssigner.java:123)
	at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.assignSplits(MySqlSourceEnumerator.java:201)
	at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.handleSplitRequest(MySqlSourceEnumerator.java:117)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$handleEventFromOperator$2(SourceCoordinator.java:230)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$9(SourceCoordinator.java:406)
	at org.apache.flink.util.ThrowableCatchingRunnable.run(ThrowableCatchingRunnable.java:40)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Error when splitting chunks for jobslink_user.jobslink_account
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitTable(MySqlSnapshotSplitAssigner.java:246)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitChunksForRemainingTables(MySqlSnapshotSplitAssigner.java:505)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	... 3 more
Caused by: java.lang.RuntimeException: Fail to analyze table in chunk splitter.
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.analyzeTable(MySqlChunkSplitter.java:154)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.splitChunks(MySqlChunkSplitter.java:117)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitTable(MySqlSnapshotSplitAssigner.java:244)
	... 7 more
Caused by: org.apache.flink.table.api.ValidationException: Incremental snapshot for tables requires primary key, but table user.account doesn't have primary key.
	at com.ververica.cdc.connectors.mysql.source.utils.ChunkUtils.getChunkKeyColumn(ChunkUtils.java:55)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.analyzeTable(MySqlChunkSplitter.java:148)
	... 9 more
16:30:43,929 INFO  org.apache.flink.runtime.jobmaster.JobMaster                 [flink-akka.actor.default-dispatcher-9]  - Trying to recover from a global failure.
org.apache.flink.util.FlinkException: Global failure triggered by OperatorCoordinator for 'Source: CDC Sourceorg.jobslink.flink.UserCDCJob -> Filter -> Flat Map -> Filter -> (Sink: Print to Std. Out, Sink: sink jobslink_account)' (operator cbc357ccb763df2852fee8c4fc7d55f2).
	at org.apache.flink.runtime.operators.coordination.OperatorCoordinatorHolder$LazyInitializedCoordinatorContext.failJob(OperatorCoordinatorHolder.java:556)
	at org.apache.flink.runtime.operators.coordination.RecreateOnResetOperatorCoordinator$QuiesceableContext.failJob(RecreateOnResetOperatorCoordinator.java:231)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinatorContext.failJob(SourceCoordinatorContext.java:316)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$9(SourceCoordinator.java:420)
	at org.apache.flink.util.ThrowableCatchingRunnable.run(ThrowableCatchingRunnable.java:40)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flink.util.FlinkRuntimeException: Chunk splitting has encountered exception
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.checkSplitterErrors(MySqlSnapshotSplitAssigner.java:522)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:278)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:298)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlHybridSplitAssigner.getNext(MySqlHybridSplitAssigner.java:123)
	at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.assignSplits(MySqlSourceEnumerator.java:201)
	at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.handleSplitRequest(MySqlSourceEnumerator.java:117)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$handleEventFromOperator$2(SourceCoordinator.java:230)
	at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$9(SourceCoordinator.java:406)
	... 9 more
Caused by: java.lang.IllegalStateException: Error when splitting chunks for jobslink_user.jobslink_account
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitTable(MySqlSnapshotSplitAssigner.java:246)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitChunksForRemainingTables(MySqlSnapshotSplitAssigner.java:505)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	... 3 more
Caused by: java.lang.RuntimeException: Fail to analyze table in chunk splitter.
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.analyzeTable(MySqlChunkSplitter.java:154)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.splitChunks(MySqlChunkSplitter.java:117)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.splitTable(MySqlSnapshotSplitAssigner.java:244)
	... 7 more
Caused by: org.apache.flink.table.api.ValidationException: Incremental snapshot for tables requires primary key, but table user.account doesn't have primary key.
	at com.ververica.cdc.connectors.mysql.source.utils.ChunkUtils.getChunkKeyColumn(ChunkUtils.java:55)
	at com.ververica.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter.analyzeTable(MySqlChunkSplitter.java:148)
	... 9 more
16:30:43,933 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph       [flink-akka.actor.default-dispatcher-9]  - Job org.flink.UserCDCJob (35780b6f0490a368fc256b228e99f7cb) switched from state RUNNING to RESTARTING.
16:30:43,936 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph       [flink-akka.actor.default-dispatcher-9]  - Source: CDC Sourceorg.flink.UserCDCJob -> Filter -> Flat Map -> Filter -> (Sink: Print to Std. Out, Sink: sink account) (1/1) (45028a2ced14e3bbe2d7cf3349defba0) switched from RUNNING to CANCELING.

Incremental snapshot for tables requires primary key, but table user.account doesn't have primary key.

表的增量快照需要主键，但表user.account没有主键。

去user库account表增加主键

添加上主键以后就可以正常执行了！！！

相关推荐
阿星AI工作室5 小时前
刘润年中大课笔记：一句话说清AI落地之战的本质
大数据·人工智能·创业创新·商业
189228048618 小时前
NY352固态MT29F32T08GWLBHD6-24QJ:B
大数据·服务器·人工智能·科技·缓存
不开大的凯20779 小时前
麦当秀AiPPT战略转向：从SaaS订阅迈向Token经济，AI办公定价模式迎来新探索
大数据·人工智能
程序鉴定师9 小时前
西安小程序制作的可靠选择与发展前景
大数据·小程序
黎阳之光9 小时前
黎阳之光：以视频孪生重构智慧医院信息化，打造高标项目核心竞争力
大数据·人工智能·物联网·算法·数字孪生
qziovv10 小时前
Git 回退场景
大数据·git·elasticsearch
清平乐的技术专栏10 小时前
【Flink学习】（五）Flink 并行度与任务链，任务运行核心原理
flink
ZeroNews内网穿透11 小时前
面向 AI 协作的本地客户端能力：ZeroNews Agent Skills
大数据·人工智能·elasticsearch
SelectDB11 小时前
Agent 时代，为什么传统的可观测方案不适用了？
大数据·数据库·数据分析
Elastic 中国社区官方博客11 小时前
快 12 倍的 Elasticsearch 向量索引：使用 GPU 和 CPU 分层部署 NVIDIA cuVS
大数据·人工智能·elasticsearch·搜索引擎·ai·全文检索·nvidia
热门推荐
01GitHub 镜像站点02Codex 接入 DeepSeek API 完整配置文档03CC-Switch & Claude 基于 Linux 服务器安装使用指南04【踩坑记录 | 第一篇】微软商店无法使用时，如何手动安装 OpenAI Codex？附`.msix`文件系统错误解决方法05几个好用的ip纯净度检测网站06裂开！ChatGPT 居然开始要手机号验证，附详细解决方法07装上就回不去了：CodeGraph 让 AI 编程效率飙升 92%，它到底做了什么？08用了半年 OpenRouter，我换到了 Ofox.ai — 两个 AI API 聚合平台的真实对比09codex app每次打开重连5次Reconnecting问题解决10【AI】2026 年具身智能模型和世界模型总结