Apache Seatunnel 抽取SQL server实时同步到Oracle-多表同步(五)

Apache Seatunnel 抽取SQL server实时同步到Oracle-多表同步(五)

上一期文章:

链接: Apache Seatunnel 抽取SQL server实时同步到Oracle-单表同步(四)

上一期单表配置信息

复制代码
[root@db1 config]# cat demo_mssql_to_oracle.conf 
env {
  job.mode = "STREAMING"
  parallelism = 1
  # 实时作业需要 checkpoint 来保证断点续传和一致性
  checkpoint.interval = 10000
}

source {
  SqlServer-CDC {
    url = "jdbc:sqlserver://192.168.56.122:1433;databaseName=AMDBdatabase;encrypt=false"
    username = "ogg_cdc"
    password = "Bai_yun123"

    # 监控的数据库名,需要与 base-url 中的库名一致
    database-names = ["AMDBdatabase"]
    # 表名格式为:数据库名.schema名.表名
    table-names = ["AMDBdatabase.dbo.tab2"]

    # 启动模式:initial 表示先全量快照,再持续增量
    startup.mode = "initial"
  }
}

sink {
  Jdbc {
    driver = "oracle.jdbc.OracleDriver"
    # 请替换为你的 Oracle 实际地址、端口和服务名
    url = "jdbc:oracle:thin:@//192.168.56.123:1521/orcl"
    username = "amdb"
    password = "amdb"

    # 使用 database + table 让连接器自动生成写入 SQL
    generate_sink_sql = true
    database = "amdb"
    table = "TAB2"

    # 指定主键,CDC 的更新/删除操作依赖它
    primary_keys = ["id"]

    #表不存在时自动创建表结构
    schema_save_mode = "CREATE_SCHEMA_WHEN_NOT_EXIST"
    data_save_mode = "APPEND_DATA"

    # 写入批次与重试参数(按需调整)
    batch_size = 1000
    max_retries = 3
  }
}
[root@db1 config]# 

这次实验中,为了避免出现上次的问题,在开启库和表的CDC功能的时候,不要把自动生成的capture和clean作业删除,capture和clean作业只在开启第一个表的CDC的时候会自动创建,后面开启第二个开始就不会再创建了,切记不要删除,否则会出现同样的情况。

开启库和表的CDC

复制代码
USE [AMDBdatabase];
GO

-- 开启数据库级 CDC
EXEC sys.sp_cdc_enable_db;
GO

--开启指定表的 CDC

USE [AMDBdatabase];
GO

EXEC sys.sp_cdc_enable_table
    @source_schema = N'dbo',
    @source_name = N'CustomerInfo',
    @role_name = NULL,
    @supports_net_changes = 0;
GO

EXEC sys.sp_cdc_enable_table
    @source_schema = N'dbo',
    @source_name = N'EmployeeInfo',
    @role_name = NULL,
    @supports_net_changes = 0;
GO

EXEC sys.sp_cdc_enable_table
    @source_schema = N'dbo',
    @source_name = N'OrderInfo',
    @role_name = NULL,
    @supports_net_changes = 0;
GO


EXEC sys.sp_cdc_enable_table
    @source_schema = N'dbo',
    @source_name = N'OrderDetail',
    @role_name = NULL,
    @supports_net_changes = 0;
GO

EXEC sys.sp_cdc_enable_table
    @source_schema = N'dbo',
    @source_name = N'ProductInfo',
    @role_name = NULL,
    @supports_net_changes = 0;
GO

--检查数据库级 CDC:
SELECT
    name,
    is_cdc_enabled
FROM sys.databases
WHERE name = N'AMDBdatabase';


--检查表级 CDC:
USE [AMDBdatabase];
GO

SELECT
    s.name AS schema_name,
    t.name AS table_name,
    t.is_tracked_by_cdc
FROM sys.tables AS t
INNER JOIN sys.schemas AS s
    ON t.schema_id = s.schema_id
WHERE t.is_ms_shipped = 0
ORDER BY s.name, t.name;

多表配置信息

复制代码
[root@db1 config]# cat demo_mssql_2_oracle_mutitable.conf 
env {
  job.mode = "STREAMING"
  parallelism = 1

  # 每 10 秒 checkpoint
  checkpoint.interval = 10000
}

source {
  SqlServer-CDC {
    plugin_output = "mssql_amdb"

    url = "jdbc:sqlserver://192.168.56.122:1433;databaseName=AMDBdatabase;encrypt=false"
    username = "ogg_cdc"
    password = "Bai_yun123"

    database-names = ["AMDBdatabase"]

    table-names = [
      "AMDBdatabase.dbo.CustomerInfo",
      "AMDBdatabase.dbo.EmployeeInfo",
      "AMDBdatabase.dbo.OrderDetail",
      "AMDBdatabase.dbo.OrderInfo",
      "AMDBdatabase.dbo.ProductInfo"
    ]

    startup.mode = "initial"
  }
}

sink {
  Jdbc {
    plugin_input = "mssql_amdb"

    driver = "oracle.jdbc.OracleDriver"
    url = "jdbc:oracle:thin:@//192.168.56.123:1521/orcl"
    username = "amdb"
    password = "amdb"

    generate_sink_sql = true

    # 五张源表分别写入同名目标表
    database = "amdb"
    table = "${table_name}"

    # 自动读取每张源表的主键元数据
    primary_keys = ["${primary_key}"]

    schema_save_mode = "CREATE_SCHEMA_WHEN_NOT_EXIST"
    data_save_mode = "APPEND_DATA"

    batch_size = 500
    batch_interval_ms = 1000
    max_retries = 3
  }
}

启动同步任务前确认目标端amdb没有这5个表

复制代码
[oracle@db1 ~]$ sqlplus amdb/amdb

SQL*Plus: Release 11.2.0.4.0 Production on Tue Oct 20 23:59:30 2026

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> select * from tab;

TNAME                          TABTYPE  CLUSTERID
------------------------------ ------- ----------
BIN$XjBRDXzjJ+HgY3s4qMBGgw==$0 TABLE
CHKPT                          TABLE
CHKPT_LOX                      TABLE
TAB2                           TABLE
TESTTAB                        TABLE

SQL> 

启动同步任务

复制代码
nohup /seatunnel_3.0/apache-seatunnel-3.0.0/bin/seatunnel.sh \
  --config /seatunnel_3.0/apache-seatunnel-3.0.0/config/demo_mssql_2_oracle_mutitable.conf  \
  -m cluster \
  -cn seatunnel \
  -n mssql2oracle_multitable \
  > /seatunnel_3.0/apache-seatunnel-3.0.0/logs/demo_mssql_2_oracle_mutitable.out 2>&1 &

[root@db1 config]# nohup /seatunnel_3.0/apache-seatunnel-3.0.0/bin/seatunnel.sh \
>   --config /seatunnel_3.0/apache-seatunnel-3.0.0/config/demo_mssql_2_oracle_mutitable.conf  \
>   -m cluster \
>   -cn seatunnel \
>   -n mssql2oracle_multitable \
>   > /seatunnel_3.0/apache-seatunnel-3.0.0/logs/demo_mssql_2_oracle_mutitable.out 2>&1 &
[1] 6542
[root@db1 config]# 

启动失败了,查看日志/seatunnel_3.0/apache-seatunnel-3.0.0/logs/demo_mssql_2_oracle_mutitable.out 如下

复制代码
[root@db1 config]# more /seatunnel_3.0/apache-seatunnel-3.0.0/logs/demo_mssql_2_oracle_mutitable.out
nohup: ignoring input
2026-10-21 00:00:29,858 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/seatunnel_3.0/apache-seatunnel-3.0.0/config/seatunnel.yaml' from System property 's
eatunnel.config'
2026-10-21 00:00:29,861 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /seatunnel_3.0/apache-seatunnel-3.0.0/config/seatunnel.yaml
2026-10-21 00:00:29,865 INFO  [o.a.s.e.c.c.SeaTunnelConfig   ] [main] - seatunnel.home is /seatunnel_3.0/apache-seatunnel-3.0.0/config
2026-10-21 00:00:29,957 INFO  [amlSeaTunnelDomConfigProcessor] [main] - Dynamic slot is enabled, the schedule strategy is set to REJECT
2026-10-21 00:00:29,957 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/seatunnel_3.0/apache-seatunnel-3.0.0/config/hazelcast.yaml' from System property 'h
azelcast.config'
2026-10-21 00:00:29,957 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /seatunnel_3.0/apache-seatunnel-3.0.0/config/hazelcast.yaml
2026-10-21 00:00:30,194 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/seatunnel_3.0/apache-seatunnel-3.0.0/config/hazelcast-client.yaml' from System prop
erty 'hazelcast.client.config'
2026-10-21 00:00:30,194 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /seatunnel_3.0/apache-seatunnel-3.0.0/config/hazelcast-client.yaml
2026-10-21 00:00:30,417 INFO  [.c.i.s.ClientInvocationService] [main] - hz.client_1 [seatunnel] [5.1] Running with 2 response threads, dynamic=true
2026-10-21 00:00:30,461 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTING
2026-10-21 00:00:30,461 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTED
2026-10-21 00:00:30,476 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Trying to connect to cluster: seatunnel
2026-10-21 00:00:30,478 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Trying to connect to [localhost]:5801
2026-10-21 00:00:30,484 WARN  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Exception during initial connection to [localhost]:5801: com.hazelcast.core.H
azelcastException: java.io.IOException: Connection refused to address localhost/127.0.0.1:5801
2026-10-21 00:00:30,485 WARN  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Unable to get live cluster connection, retry in 1000 ms, attempt: 1, cluster 
connect timeout: 3000 ms, max backoff: 30000 ms
2026-10-21 00:00:31,489 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Trying to connect to [localhost]:5801
2026-10-21 00:00:31,489 WARN  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Exception during initial connection to [localhost]:5801: com.hazelcast.core.H
azelcastException: java.io.IOException: Connection refused to address localhost/127.0.0.1:5801
2026-10-21 00:00:31,489 WARN  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Unable to get live cluster connection, retry in 1050 ms, attempt: 2, cluster 
connect timeout: 3000 ms, max backoff: 30000 ms
2026-10-21 00:00:32,541 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Trying to connect to [localhost]:5801
2026-10-21 00:00:32,541 WARN  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Exception during initial connection to [localhost]:5801: com.hazelcast.core.H
azelcastException: java.io.IOException: Connection refused to address localhost/127.0.0.1:5801
2026-10-21 00:00:32,542 WARN  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Unable to get live cluster connection, retry in 936 ms, attempt: 3, cluster c
onnect timeout: 3000 ms, max backoff: 30000 ms
2026-10-21 00:00:33,479 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Trying to connect to [localhost]:5801
2026-10-21 00:00:33,479 WARN  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Exception during initial connection to [localhost]:5801: com.hazelcast.core.H
azelcastException: java.io.IOException: Connection refused to address localhost/127.0.0.1:5801
2026-10-21 00:00:33,479 WARN  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Unable to get live cluster connection, cluster connect timeout (3000 ms) is r
eached. Attempt 4.
2026-10-21 00:00:33,480 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Unable to connect to any address from the cluster with name: seatunnel. The f
ollowing addresses were tried: [[localhost]:5801]
2026-10-21 00:00:33,480 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN
2026-10-21 00:00:33,482 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN
2026-10-21 00:00:33,482 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - 

===============================================================================


2026-10-21 00:00:33,482 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Fatal Error, 

2026-10-21 00:00:33,482 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Please submit bug report in https://github.com/apache/seatunnel/issues

2026-10-21 00:00:33,484 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Reason:SeaTunnel job executed failed 

2026-10-21 00:00:33,487 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job exe
cuted failed
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:291)
        at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
        at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:41)
Caused by: java.lang.IllegalStateException: Unable to connect to any cluster.
        at com.hazelcast.client.impl.connection.tcp.TcpClientConnectionManager.doConnectToCluster(TcpClientConnectionManager.java:433)
        at com.hazelcast.client.impl.connection.tcp.TcpClientConnectionManager.connectToCluster(TcpClientConnectionManager.java:372)
        at com.hazelcast.client.impl.clientside.HazelcastClientInstanceImpl.start(HazelcastClientInstanceImpl.java:381)
        at com.hazelcast.client.HazelcastClient.constructHazelcastClient(HazelcastClient.java:460)
        at com.hazelcast.client.HazelcastClient.newHazelcastClientInternal(HazelcastClient.java:416)
        at com.hazelcast.client.HazelcastClient.newHazelcastClient(HazelcastClient.java:136)
        at org.apache.seatunnel.engine.client.SeaTunnelHazelcastClient.<init>(SeaTunnelHazelcastClient.java:47)
        at org.apache.seatunnel.engine.client.SeaTunnelClient.<init>(SeaTunnelClient.java:57)
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:113)
        ... 2 more
 
2026-10-21 00:00:33,487 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - 
===============================================================================



Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:291)
        at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
        at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:41)
Caused by: java.lang.IllegalStateException: Unable to connect to any cluster.
        at com.hazelcast.client.impl.connection.tcp.TcpClientConnectionManager.doConnectToCluster(TcpClientConnectionManager.java:433)
        at com.hazelcast.client.impl.connection.tcp.TcpClientConnectionManager.connectToCluster(TcpClientConnectionManager.java:372)
        at com.hazelcast.client.impl.clientside.HazelcastClientInstanceImpl.start(HazelcastClientInstanceImpl.java:381)
        at com.hazelcast.client.HazelcastClient.constructHazelcastClient(HazelcastClient.java:460)
        at com.hazelcast.client.HazelcastClient.newHazelcastClientInternal(HazelcastClient.java:416)
        at com.hazelcast.client.HazelcastClient.newHazelcastClient(HazelcastClient.java:136)
        at org.apache.seatunnel.engine.client.SeaTunnelHazelcastClient.<init>(SeaTunnelHazelcastClient.java:47)
        at org.apache.seatunnel.engine.client.SeaTunnelClient.<init>(SeaTunnelClient.java:57)
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:113)
        ... 2 more
[root@db1 config]# 
[root@db1 config]# ps -ef |grep seatunnel
root      6733  5182  0 00:02 pts/0    00:00:00 grep --color=auto seatunnel

解决方法,启动seatunnel集群服务。

复制代码
/seatunnel_3.0/apache-seatunnel-3.0.0/bin/seatunnel-cluster.sh 2>&1 &

再次启动同步任务,正常RUNNING

查看同步进程状态:/seatunnel_3.0/apache-seatunnel-3.0.0/bin/seatunnel.sh -l

复制代码
[root@db1 config]# nohup /seatunnel_3.0/apache-seatunnel-3.0.0/bin/seatunnel.sh   --config /seatunnel_3.0/apache-seatunnel-3.0.0/config/demo_mssql_2_oracle_mutitable.conf    -m cluster   -cn seatunnel   -n mssql2oracle_multitable   > /seatunnel_3.0/apache-seatunnel-3.0.0/logs/demo_mssql_2_oracle_mutitable.out 2>&1 &
[2] 7010
[root@db1 config]# 
[root@db1 config]# 
[root@db1 config]# 
[root@db1 config]# /seatunnel_3.0/apache-seatunnel-3.0.0/bin/seatunnel.sh -l
2026-10-21 00:05:03,544 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/seatunnel_3.0/apache-seatunnel-3.0.0/config/seatunnel.yaml' from System property 'seatunnel.config'
2026-10-21 00:05:03,546 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /seatunnel_3.0/apache-seatunnel-3.0.0/config/seatunnel.yaml
2026-10-21 00:05:03,550 INFO  [o.a.s.e.c.c.SeaTunnelConfig   ] [main] - seatunnel.home is /seatunnel_3.0/apache-seatunnel-3.0.0/config
2026-10-21 00:05:03,645 INFO  [amlSeaTunnelDomConfigProcessor] [main] - Dynamic slot is enabled, the schedule strategy is set to REJECT
2026-10-21 00:05:03,645 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/seatunnel_3.0/apache-seatunnel-3.0.0/config/hazelcast.yaml' from System property 'hazelcast.config'
2026-10-21 00:05:03,645 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /seatunnel_3.0/apache-seatunnel-3.0.0/config/hazelcast.yaml
2026-10-21 00:05:04,041 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/seatunnel_3.0/apache-seatunnel-3.0.0/config/hazelcast-client.yaml' from System property 'hazelcast.client.config'
2026-10-21 00:05:04,041 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /seatunnel_3.0/apache-seatunnel-3.0.0/config/hazelcast-client.yaml
2026-10-21 00:05:04,339 INFO  [.c.i.s.ClientInvocationService] [main] - hz.client_1 [seatunnel] [5.1] Running with 2 response threads, dynamic=true
2026-10-21 00:05:04,410 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTING
2026-10-21 00:05:04,410 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTED
2026-10-21 00:05:04,443 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Trying to connect to cluster: seatunnel
2026-10-21 00:05:04,448 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Trying to connect to [localhost]:5801
2026-10-21 00:05:04,474 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_CONNECTED
2026-10-21 00:05:04,475 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Authenticated with server [localhost]:5801:2f0a0cf5-e44d-4ecf-85fc-0dd26de59916, server version: 5.1, local address: /127.0.0.1:58784
2026-10-21 00:05:04,476 INFO  [c.h.i.d.Diagnostics           ] [main] - hz.client_1 [seatunnel] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2026-10-21 00:05:04,495 INFO  [c.h.c.i.s.ClientClusterService] [hz.client_1.event-3] - hz.client_1 [seatunnel] [5.1] 

Members [1] {
        Member [localhost]:5801 - 2f0a0cf5-e44d-4ecf-85fc-0dd26de59916 [master node]
}

2026-10-21 00:05:04,576 INFO  [.c.i.s.ClientStatisticsService] [main] - Client statistics is enabled with period 5 seconds.
Job ID               Job Name                 Job Status  Submit Time              Start Time               Finished Time            
-------------------  -----------------------  ----------  -----------------------  -----------------------  -----------------------  
1165138622970593281  mssql2oracle_multitable  RUNNING     2026-10-21 00:04:55.962  2026-10-21 00:04:57.478  

2026-10-21 00:05:04,984 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN
2026-10-21 00:05:04,993 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Removed connection to endpoint: [localhost]:5801:2f0a0cf5-e44d-4ecf-85fc-0dd26de59916, connection: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/127.0.0.1:58784->localhost/127.0.0.1:5801}, remoteAddress=[localhost]:5801, lastReadTime=2026-10-21 00:05:04.886, lastWriteTime=2026-10-21 00:05:04.716, closedTime=2026-10-21 00:05:04.989, connected server version=5.1}
2026-10-21 00:05:04,993 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED
2026-10-21 00:05:04,997 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN
2026-10-21 00:05:05,002 INFO  [.s.a.m.MetadataProviderManager] [main] - MetadataProvider closed
2026-10-21 00:05:05,003 INFO  [s.c.s.s.c.ClientExecuteCommand] [main] - Closed SeaTunnel client......
[root@db1 config]# 

查看同步日志/seatunnel_3.0/apache-seatunnel-3.0.0/logs/demo_mssql_2_oracle_mutitable.out ,正常同步并初始化了数据

复制代码
***********************************************
           Job Progress Information
***********************************************
Job Id                    : 1165138622970593281
Read Count So Far         :              250000
Write Attempt Count So Far:              250000
Write Committed Count So Far:              250000
Commit Rate               :             100.00%
Average Read Count        :              4166/s
Average Write Attempt Count:              4166/s
Average Write Committed Count:              4166/s
Last Statistic Time       : 2026-10-21 00:04:57
Current Statistic Time    : 2026-10-21 00:05:57
***********************************************

目标端查验5个表已经正常同步并初始化数据

复制代码
SQL> select * from tab;

TNAME                          TABTYPE  CLUSTERID
------------------------------ ------- ----------
CHKPT                          TABLE
CHKPT_LOX                      TABLE
CustomerInfo                   TABLE
EmployeeInfo                   TABLE
OrderDetail                    TABLE
OrderInfo                      TABLE
ProductInfo                    TABLE
TAB2                           TABLE
TESTTAB                        TABLE

SQL> select count(*) from "CustomerInfo";

  COUNT(*)
----------
     50000

SQL> select count(*) from "EmployeeInfo";

  COUNT(*)
----------
     50000

细心的你已经发现了,在目标端查询表名字的时候如果不加双引号,无法识别到表名字,需要加双引号才能识别到表名字。正常情况下在Oracle存储的表名字均显示大写,现在我们看到表名字与源端保持一致,有大小写,原因应该是seatunnel的配置文件中 ,目标端表名字了加了双引号。

以下让AI生成了一批DML测试语句

执行 INSERT 新增测试数据

复制代码
USE [AMDBdatabase];
GO

-- 1. CustomerInfo:新增客户
SET IDENTITY_INSERT dbo.CustomerInfo ON;

INSERT INTO dbo.CustomerInfo
    (CustomerID, CustomerCode, CustomerName, Gender,
     Age, Phone, City)
VALUES
    (990001, 'CDC_CUST_990001', N'CDC测试客户990001', 'M',
     30, '13900009901', N'上海');

SET IDENTITY_INSERT dbo.CustomerInfo OFF;
GO

-- 2. EmployeeInfo:新增员工
SET IDENTITY_INSERT dbo.EmployeeInfo ON;

INSERT INTO dbo.EmployeeInfo
    (EmployeeID, EmployeeCode, EmployeeName, Gender,
     Department, JobTitle, Salary, HireDate)
VALUES
    (990002, 'CDC_EMP_990002', N'CDC测试员工990002', 'F',
     N'技术部', N'数据库工程师', 18000.00, '2025-01-15');

SET IDENTITY_INSERT dbo.EmployeeInfo OFF;
GO

-- 3. ProductInfo:新增商品
SET IDENTITY_INSERT dbo.ProductInfo ON;

INSERT INTO dbo.ProductInfo
    (ProductID, ProductCode, ProductName, Category,
     UnitPrice, StockQty, ProductStatus)
VALUES
    (990003, 'CDC_PROD_990003', N'CDC测试商品990003', N'测试分类',
     99.90, 100, 1);

SET IDENTITY_INSERT dbo.ProductInfo OFF;
GO

-- 4. OrderInfo:新增订单
SET IDENTITY_INSERT dbo.OrderInfo ON;

INSERT INTO dbo.OrderInfo
    (OrderID, OrderNo, CustomerID, OrderDate,
     TotalAmount, OrderStatus, PayMethod, Remark)
VALUES
    (990004, 'CDC_ORDER_990004', 990001, SYSDATETIME(),
     199.80, 1, N'测试支付', N'CDC实时同步测试订单');

SET IDENTITY_INSERT dbo.OrderInfo OFF;
GO

-- 5. OrderDetail:新增订单明细
SET IDENTITY_INSERT dbo.OrderDetail ON;

INSERT INTO dbo.OrderDetail
    (DetailID, OrderID, ProductID, Quantity,
     UnitPrice, DiscountRate, LineAmount)
VALUES
    (990005, 990004, 990003, 2,
     99.90, 0.00, 199.80);

SET IDENTITY_INSERT dbo.OrderDetail OFF;
GO

执行 UPDATE 修改测试

复制代码
USE [AMDBdatabase];
GO

-- 1. 修改客户信息
UPDATE dbo.CustomerInfo
SET CustomerName = N'CDC测试客户990001_已修改',
    Age = 31,
    City = N'北京'
WHERE CustomerID = 990001;

-- 2. 修改员工信息
UPDATE dbo.EmployeeInfo
SET Department = N'运维部',
    JobTitle = N'高级数据库工程师',
    Salary = 20000.00
WHERE EmployeeID = 990002;

-- 3. 修改商品信息
UPDATE dbo.ProductInfo
SET ProductName = N'CDC测试商品990003_已修改',
    UnitPrice = 109.90,
    StockQty = 88
WHERE ProductID = 990003;

-- 4. 修改订单信息
UPDATE dbo.OrderInfo
SET TotalAmount = 219.80,
    OrderStatus = 2,
    Remark = N'CDC UPDATE 测试成功'
WHERE OrderID = 990004;

-- 5. 修改订单明细
UPDATE dbo.OrderDetail
SET Quantity = 2,
    UnitPrice = 109.90,
    LineAmount = 219.80
WHERE DetailID = 990005;
GO

执行 DELETE 删除测试

复制代码
USE [AMDBdatabase];
GO

-- 1. 删除订单明细
DELETE FROM dbo.OrderDetail
WHERE DetailID = 990005;

-- 2. 删除订单
DELETE FROM dbo.OrderInfo
WHERE OrderID = 990004;

-- 3. 删除客户
DELETE FROM dbo.CustomerInfo
WHERE CustomerID = 990001;

-- 4. 删除商品
DELETE FROM dbo.ProductInfo
WHERE ProductID = 990003;

-- 5. 删除员工
DELETE FROM dbo.EmployeeInfo
WHERE EmployeeID = 990002;
GO

验证 SQL Server 端测试结果

复制代码
USE [AMDBdatabase];
GO

SELECT * FROM dbo.CustomerInfo WHERE CustomerID = 990001;
SELECT * FROM dbo.EmployeeInfo WHERE EmployeeID = 990002;
SELECT * FROM dbo.ProductInfo WHERE ProductID = 990003;
SELECT * FROM dbo.OrderInfo WHERE OrderID = 990004;
SELECT * FROM dbo.OrderDetail WHERE DetailID = 990005;
GO

验证 Oracle 端实时同步.

END

seatunnel DML同步数据的统计信息

前面的步骤中,可以通过启动任务时指定的日志文件tail -100f /seatunnel_3.0/apache-seatunnel-3.0.0/logs/demo_mssql_2_oracle_mutitable.out 查看到统计信息,但是无法统计到insert ,update ,delete分别做了多少操作,只是一个总数。

复制代码
2026-10-21 03:23:37,984 INFO  [o.a.s.e.c.j.JobMetricsRunner  ] [job-metrics-runner-1165187367112278017] - 
***********************************************
           Job Progress Information
***********************************************
Job Id                    : 1165187367112278017
Read Count So Far         :              250000
Write Attempt Count So Far:              250000
Write Committed Count So Far:              250000
Commit Rate               :             100.00%
Average Read Count        :                 0/s
Average Write Attempt Count:                 0/s
Average Write Committed Count:                 0/s
Last Statistic Time       : 2026-10-21 03:22:37
Current Statistic Time    : 2026-10-21 03:23:37
***********************************************

insert 插入一条数据统计数据+1.

复制代码
2026-10-21 03:24:37,985 INFO  [o.a.s.e.c.j.JobMetricsRunner  ] [job-metrics-runner-1165187367112278017] - 
***********************************************
           Job Progress Information
***********************************************
Job Id                    : 1165187367112278017
Read Count So Far         :              250001
Write Attempt Count So Far:              250001
Write Committed Count So Far:              250001
Commit Rate               :             100.00%
Average Read Count        :                 0/s
Average Write Attempt Count:                 0/s
Average Write Committed Count:                 0/s
Last Statistic Time       : 2026-10-21 03:23:37
Current Statistic Time    : 2026-10-21 03:24:37

update 更新一条数据统计数据+2,也就是说seatunnel在更新数据的时候会先把目标库的数据做一次delete操作,然后再插入一条更新后的数据,所以统计信息时+2.

复制代码
2026-10-21 03:26:37,983 INFO  [o.a.s.e.c.j.JobMetricsRunner  ] [job-metrics-runner-1165187367112278017] - 
***********************************************
           Job Progress Information
***********************************************
Job Id                    : 1165187367112278017
Read Count So Far         :              250003
Write Attempt Count So Far:              250003
Write Committed Count So Far:              250003
Commit Rate               :             100.00%
Average Read Count        :                 0/s
Average Write Attempt Count:                 0/s
Average Write Committed Count:                 0/s
Last Statistic Time       : 2026-10-21 03:25:37
Current Statistic Time    : 2026-10-21 03:26:37
***********************************************

delete 删除一条数据统计信息+1

复制代码
2026-10-21 03:32:37,983 INFO  [o.a.s.e.c.j.JobMetricsRunner  ] [job-metrics-runner-1165187367112278017] - 
***********************************************
           Job Progress Information
***********************************************
Job Id                    : 1165187367112278017
Read Count So Far         :              250004
Write Attempt Count So Far:              250004
Write Committed Count So Far:              250004
Commit Rate               :             100.00%
Average Read Count        :                 0/s
Average Write Attempt Count:                 0/s
Average Write Committed Count:                 0/s
Last Statistic Time       : 2026-10-21 03:31:37
Current Statistic Time    : 2026-10-21 03:32:37
***********************************************

待验证功能

建议下一步再验证 4 个生产环境关键场景

  1. 断点续传:停止 SeaTunnel 作业,再启动,确认不会遗漏变更。
  2. 持续运行:连续执行大量 INSERT、UPDATE、DELETE,观察同步延迟和任务稳定性。
  3. 数据库重启:分别重启 SQL Server 或 SeaTunnel Engine,确认作业能够恢复。(重启SQL server数据库之后,同步任务没有中断)
  4. CDC 清理与磁盘增长:观察 SQL Server CDC 捕获作业、清理作业、事务日志及 CDC 变更表的增长情况。

预留问题:

针对目标表名字大小写的问题,目前未找到解决方法,seatunnel的配置文件无法针对多个表进行大写的自动转换,有一个变通的解决方法是在启动同步任务前,手工在目标库创建目标表名字。

相关推荐
神也畏惧菠萝44 分钟前
Coze 工作流数据库 Day06:架构与高级循环控制的深度技术分析
数据库·架构·llm·coze·coze数据库·数据库流程结构
麦壳饼1 小时前
SonnetDB:空间函数别名的参数与兼容边界
数据库·sonnetdb
han68891 小时前
jmeter操作数据库
数据库·jmeter
星马梦缘1 小时前
人工智能学院游戏设计大赛决赛 · 执行提纲
数据库·人工智能·游戏
eleven_4u2 小时前
MySQL引擎层是如何工作的
数据库·mysql
love静思冥想2 小时前
CLOB中JSON查询agent字段值(Oracle11g)
sql·oracle·json
adinnet20263 小时前
企业级 NL2SQL 的重要基础:数据库 Schema 自动理解与元数据管理
数据库
IvorySQL3 小时前
PostgreSQL 日报| OpenCode 遭黑客入侵 API 密钥失窃(10 月 20 日)
数据库·postgresql·区块链
Elastic 中国社区官方博客3 小时前
控制告警如何触发、恢复和发送通知:Elastic 可观测性中的 Kibana 通用告警
大数据·数据库·安全·elasticsearch·搜索引擎·全文检索·es