作者：Ricardo Ferreira
编译：Debra Chen
假设你把客户数据存储在 Amazon DynamoDB 中，作为应用的唯一数据源（single source of truth）。但如今你需要同样的数据放在 Redis 里，进行快速缓存、实时分析，或是为向量搜索推荐引擎赋能。举例来说，DynamoDB 中这张名为 mySourceTable 的表当前包含两条数据记录。
那么这个任务的挑战在哪呢？如何在不大写一堆代码的情况下，让两边保持同步？
就我个人而言，我最理想的方案是使用 Redis Data Integration（RDI），但遗憾的是，在撰写本文时，RDI 还不支持把 Amazon DynamoDB 作为数据源。这就让开发者只剩下两个选择：自己编写复杂的数据集成管道，或者投入昂贵（而且往往溢价过高）的 ETL 解决方案。这两个选项都不怎么让人满意。
但还有另一条路。
那就是 Apache SeaTunnel------一个开源数据集成项目，它让数据同步变得像写一个配置文件一样简单。在这篇博客中，我会深入介绍这个项目，并一步步讲解如何把数据从 Amazon DynamoDB 同步到 Redis。
Apache SeaTunnel：你的开源数据管道好帮手
Apache SeaTunnel 是一个多模态、高性能、分布式的海量数据集成工具，同时支持批处理和流式两种模式。你可以把它想象成你数据系统的"万能翻译官"------它精通 DynamoDB、Redis 以及其他 100 多种数据系统。
在这个场景下，SeaTunnel 的突出优势在于：
- 零代码：用简单的配置文件即可定义你的数据管道
- 分布式：当数据增长时，同步可横向扩展
- 容错：内置 checkpoint 机制，确保数据不丢失
你可以用两种不同的配置方式使用 SeaTunnel：单机模式 （standalone），用于一次性的数据搬运；或者集群模式（cluster），它能搭建一套常驻在线的数据管道基础设施，随时准备执行新任务。
方案一：快速上手（单机模式）
这个方案非常适合一次性迁移、本地开发或定时批处理任务。一条命令，搞定一切。
第一步，创建你的管道配置文件（jobs/my.config）：
env {
parallelism = 1
job.mode = "BATCH"
}
source {
Amazondynamodb {
url = "http://dynamodb.us-east-1.amazonaws.com"
region = "us-east-1"
access_key_id = "YOUR_ACCESS_KEY"
secret_access_key = "YOUR_SECRET_KEY"
table = "mySourceTable"
schema = {
fields {
customerId = int
customerName = string
address = string
}
}
}
}
sink {
Redis {
host = host.docker.internal # 或者你的 Redis 主机地址
port = 6379
support_custom_key = true
key = "customer:{customerId}"
data_type = hash
}
}用一条 Docker 命令即可运行：
docker run --rm -it \
-v $(pwd)/jobs:/config \
apache/seatunnel:2.3.12 \
./bin/seatunnel.sh -m local -c /config/my.config这条命令会启动一个新容器，里面运行一个单机模式 SeaTunnel 实例，执行 my.config 文件中描述的任务。你会看到类似下面的输出：
2026-01-09 16:57:50,786 INFO [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/opt/seatunnel/config/seatunnel.yaml' from System property 'seatunnel.config'
2026-01-09 16:57:50,788 INFO [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /opt/seatunnel/config/seatunnel.yaml
2026-01-09 16:57:50,789 INFO [o.a.s.e.c.c.SeaTunnelConfig ] [main] - seatunnel.home is /opt/seatunnel
2026-01-09 16:57:50,830 INFO [amlSeaTunnelDomConfigProcessor] [main] - Dynamic slot is enabled, the schedule strategy is set to REJECT
2026-01-09 16:57:50,830 INFO [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/opt/seatunnel/config/hazelcast.yaml' from System property 'hazelcast.config'
2026-01-09 16:57:50,830 INFO [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /opt/seatunnel/config/hazelcast.yaml
2026-01-09 16:57:50,962 INFO [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/opt/seatunnel/config/hazelcast-client.yaml' from System property 'hazelcast.client.config'
2026-01-09 16:57:50,962 INFO [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /opt/seatunnel/config/hazelcast-client.yaml
2026-01-09 16:57:50,985 WARN [c.h.i.AddressPicker ] [main] - [LOCAL] [seatunnel-420804] [5.1] You configured your member address as host name. Please be aware of that your dns can be spoofed. Make sure that your dns configurations are correct.
2026-01-09 16:57:50,985 INFO [c.h.i.AddressPicker ] [main] - [LOCAL] [seatunnel-420804] [5.1] Resolving domain name 'localhost' to address(es): [127.0.0.1, 0:0:0:0:0:0:0:1]
2026-01-09 16:57:50,986 INFO [c.h.i.AddressPicker ] [main] - [LOCAL] [seatunnel-420804] [5.1] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1]
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
2026-01-09 16:57:51,189 INFO [o.a.s.e.s.SeaTunnelServer ] [main] - SeaTunnel server start...
2026-01-09 16:57:51,191 INFO [c.h.system ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Based on Hazelcast IMDG version: 5.1.0 (20220228 - 21f20e7)
2026-01-09 16:57:51,191 INFO [c.h.system ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Cluster name: seatunnel-420804
2026-01-09 16:57:51,191 INFO [c.h.system ] [main] - [localhost]:5801 [seatunnel-420804] [5.1]
_____ _____ _
/ ___| |_ _| | |
\ `--. ___ __ _ | | _ _ _ __ _ __ ___ | |
`--. \ / _ \ / _` | | | | | | || '_ \ | '_ \ / _ \| |
/\__/ /| __/| (_| | | | | |_| || | | || | | || __/| |
\____/ \___| \__,_| \_/ \__,_||_| |_||_| |_| \___||_|
2026-01-09 16:57:51,191 INFO [c.h.system ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Copyright © 2021-2024 The Apache Software Foundation. Apache SeaTunnel, SeaTunnel, and its feather logo are trademarks of The Apache Software Foundation.
2026-01-09 16:57:51,191 INFO [c.h.system ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Integrity Checker is disabled. Fail-fast on corrupted executables will not be performed.
To enable integrity checker do one of the following:
- Change member config using Java API: config.setIntegrityCheckerEnabled(true);
- Change XML/YAML configuration property: Set hazelcast.integrity-checker.enabled to true
- Add system property: -Dhz.integritychecker.enabled=true (for Hazelcast embedded, works only when loading config via Config.load)
- Add environment variable: HZ_INTEGRITYCHECKER_ENABLED=true (recommended when running container image. For Hazelcast embedded, works only when loading config via Config.load)
2026-01-09 16:57:51,193 INFO [c.h.system ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] The Jet engine is disabled.
To enable the Jet engine on the members, do one of the following:
- Change member config using Java API: config.getJetConfig().setEnabled(true)
- Change XML/YAML configuration property: Set hazelcast.jet.enabled to true
- Add system property: -Dhz.jet.enabled=true (for Hazelcast embedded, works only when loading config via Config.load)
- Add environment variable: HZ_JET_ENABLED=true (recommended when running container image. For Hazelcast embedded, works only when loading config via Config.load)
2026-01-09 16:57:51,331 INFO [c.h.s.security ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Enable DEBUG/FINE log level for log category com.hazelcast.system.security or use -Dhazelcast.security.recommendations system property to see 🔒 security recommendations and the status of current config.
2026-01-09 16:57:51,392 INFO [o.a.s.e.s.SeaTunnelNodeContext] [main] - Using LiteNodeDropOutTcpIpJoiner TCP/IP discovery
2026-01-09 16:57:51,393 WARN [c.h.c.CPSubsystem ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] CP Subsystem is not enabled. CP data structures will operate in UNSAFE mode! Please note that UNSAFE mode will not provide strong consistency guarantees.
2026-01-09 16:57:51,462 INFO [.c.c.DefaultClassLoaderService] [main] - start classloader service with cache mode
2026-01-09 16:57:51,464 INFO [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/opt/seatunnel/config/seatunnel.yaml' from System property 'seatunnel.config'
2026-01-09 16:57:51,464 INFO [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /opt/seatunnel/config/seatunnel.yaml
2026-01-09 16:57:51,466 INFO [amlSeaTunnelDomConfigProcessor] [main] - Dynamic slot is enabled, the schedule strategy is set to REJECT
2026-01-09 16:57:51,466 INFO [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/opt/seatunnel/config/hazelcast.yaml' from System property 'hazelcast.config'
2026-01-09 16:57:51,466 INFO [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /opt/seatunnel/config/hazelcast.yaml
2026-01-09 16:57:51,471 WARN [o.a.s.e.s.TaskExecutionService] [pool-3-thread-1] - [localhost]:5801 [seatunnel-420804] [5.1] The Node is not ready yet, Node state STARTING,looking forward to the next scheduling
2026-01-09 16:57:51,471 INFO [o.a.s.e.s.TaskExecutionService] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Created new BusWork : 1257526338
2026-01-09 16:57:51,474 WARN [a.s.e.s.s.s.DefaultSlotService] [hz.main.seaTunnel.slotService.thread] - failed send heartbeat to resource manager, will retry later. this address: [localhost]:5801
2026-01-09 16:57:51,475 INFO [o.a.s.e.s.CoordinatorService ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Start pending job schedule thread
2026-01-09 16:57:51,554 WARN [o.a.h.u.NativeCodeLoader ] [main] - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2026-01-09 16:57:51,607 INFO [o.a.s.e.s.CoordinatorService ] [pool-7-thread-1] - [localhost]:5801 [seatunnel-420804] [5.1]
***********************************************
CoordinatorService Thread Pool Status
***********************************************
activeCount : 1
corePoolSize : 10
maximumPoolSize : 2147483647
poolSize : 1
completedTaskCount : 0
taskCount : 1
***********************************************
2026-01-09 16:57:51,612 INFO [o.a.s.e.s.JettyService ] [main] - SeaTunnel REST service will start on port 8080
2026-01-09 16:57:51,617 INFO [o.a.s.s.o.e.j.u.log ] [main] - Logging initialized @1069ms to org.apache.seatunnel.shade.org.eclipse.jetty.util.log.Slf4jLog
2026-01-09 16:57:51,643 WARN [a.s.s.o.e.j.s.h.ContextHandler] [main] - Empty contextPath
2026-01-09 16:57:51,652 INFO [o.a.s.s.o.e.j.s.Server ] [main] - jetty-9.4.56.v20240826; built: 2024-08-26T17:15:05.868Z; git: ec6782ff5ead824dabdcf47fa98f90a4aedff401; jvm 1.8.0_342-b07
2026-01-09 16:57:51,665 INFO [o.a.s.s.o.e.j.s.session ] [main] - DefaultSessionIdManager workerName=node0
2026-01-09 16:57:51,665 INFO [o.a.s.s.o.e.j.s.session ] [main] - No SessionScavenger set, using defaults
2026-01-09 16:57:51,666 INFO [o.a.s.s.o.e.j.s.session ] [main] - node0 Scavenging every 660000ms
2026-01-09 16:57:51,727 INFO [a.s.s.o.e.j.s.h.ContextHandler] [main] - Started o.a.s.s.o.e.j.s.ServletContextHandler@7069f076{/,null,AVAILABLE}
2026-01-09 16:57:51,731 INFO [.s.s.o.e.j.s.AbstractConnector] [main] - Started ServerConnector@6e31d989{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2026-01-09 16:57:51,732 INFO [o.a.s.s.o.e.j.s.Server ] [main] - Started @1184ms
2026-01-09 16:57:51,747 INFO [c.h.i.d.Diagnostics ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2026-01-09 16:57:51,749 INFO [c.h.c.LifecycleService ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] [localhost]:5801 is STARTING
Members {size:1, ver:1} [
Member [localhost]:5801 - 579e9374-7d47-4d91-80d5-c7a9155d42d0 [master node] [active master] this
]
2026-01-09 16:57:52,775 INFO [o.a.s.e.s.CoordinatorService ] [pool-5-thread-1] - [localhost]:5801 [seatunnel-420804] [5.1] This node become a new active master node, begin init coordinator service
2026-01-09 16:57:52,779 INFO [c.h.c.LifecycleService ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] [localhost]:5801 is STARTED
2026-01-09 16:57:52,797 INFO [o.a.s.e.s.CoordinatorService ] [pool-5-thread-1] - [localhost]:5801 [seatunnel-420804] [5.1] Loaded event handlers: [org.apache.seatunnel.api.event.LoggingEventHandler@1ebbcf65]
2026-01-09 16:57:52,803 INFO [.c.i.s.ClientInvocationService] [main] - hz.client_1 [seatunnel-420804] [5.1] Running with 2 response threads, dynamic=true
2026-01-09 16:57:52,807 INFO [.h.i.p.i.PartitionStateManager] [seatunnel-coordinator-service-1] - [localhost]:5801 [seatunnel-420804] [5.1] Initializing cluster partition table arrangement...
2026-01-09 16:57:52,811 INFO [c.h.c.LifecycleService ] [main] - hz.client_1 [seatunnel-420804] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTING
2026-01-09 16:57:52,811 INFO [c.h.c.LifecycleService ] [main] - hz.client_1 [seatunnel-420804] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTED
2026-01-09 16:57:52,814 INFO [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-420804] [5.1] Trying to connect to cluster: seatunnel-420804
2026-01-09 16:57:52,815 INFO [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-420804] [5.1] Trying to connect to [localhost]:5801
2026-01-09 16:57:52,823 INFO [.p.t.AuthenticationMessageTask] [hz.main.priority-generic-operation.thread-0] - [localhost]:5801 [seatunnel-420804] [5.1] Received auth from Connection[id=1, /127.0.0.1:5801->/127.0.0.1:45255, qualifier=null, endpoint=[127.0.0.1]:45255, remoteUuid=3be41dca-f0e7-4603-ad27-4e5b3aecbb4a, alive=true, connectionType=JVM, planeIndex=-1], successfully authenticated, clientUuid: 3be41dca-f0e7-4603-ad27-4e5b3aecbb4a, client name: hz.client_1, client version: 5.1
2026-01-09 16:57:52,824 INFO [c.h.c.LifecycleService ] [main] - hz.client_1 [seatunnel-420804] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_CONNECTED
2026-01-09 16:57:52,824 INFO [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-420804] [5.1] Authenticated with server [localhost]:5801:579e9374-7d47-4d91-80d5-c7a9155d42d0, server version: 5.1, local address: /127.0.0.1:45255
2026-01-09 16:57:52,825 INFO [c.h.i.d.Diagnostics ] [main] - hz.client_1 [seatunnel-420804] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2026-01-09 16:57:52,829 INFO [c.h.c.i.s.ClientClusterService] [hz.client_1.event-10] - hz.client_1 [seatunnel-420804] [5.1]
Members [1] {
Member [localhost]:5801 - 579e9374-7d47-4d91-80d5-c7a9155d42d0 [master node]
}
2026-01-09 16:57:52,841 INFO [.c.i.s.ClientStatisticsService] [main] - Client statistics is enabled with period 5 seconds.
2026-01-09 16:57:52,858 INFO [o.a.s.c.s.u.ConfigBuilder ] [main] - Loading config file from path: /config/my.config
2026-01-09 16:57:52,951 INFO [o.a.s.c.s.u.ConfigShadeUtils ] [main] - Load config shade spi: [base64]
2026-01-09 16:57:52,967 INFO [o.a.s.c.s.u.ConfigBuilder ] [main] - Parsed config file:
{
"env" : {
"parallelism" : 1,
"job.mode" : "BATCH"
},
"source" : [
{
"url" : "http://dynamodb.us-east-1.amazonaws.com",
"region" : "us-east-1",
"access_key_id" : "<REDACTED>",
"secret_access_key" : "<REDACTED>",
"table" : "mySourceTable",
"schema" : {
"fields" : {
"customerId" : "int",
"customerName" : "string",
"address" : "string"
}
},
"plugin_name" : "Amazondynamodb"
}
],
"sink" : [
{
"host" : "host.docker.internal",
"port" : 6379,
"support_custom_key" : true,
"key" : "customer:{customerId}",
"data_type" : "hash",
"plugin_name" : "Redis"
}
]
}
2026-01-09 16:57:52,971 INFO [p.MultipleTableJobConfigParser] [main] - add common jar in plugins :[]
2026-01-09 16:57:52,978 INFO [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /opt/seatunnel/connectors
2026-01-09 16:57:52,980 INFO [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='Amazondynamodb'} at: [file:/opt/seatunnel/connectors/connector-amazondynamodb-2.3.12.jar]
2026-01-09 16:57:52,980 INFO [.s.p.d.AbstractPluginDiscovery] [main] - find connector jar and dependency for PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='Amazondynamodb'}: [file:/opt/seatunnel/connectors/connector-amazondynamodb-2.3.12.jar]
2026-01-09 16:57:52,982 INFO [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /opt/seatunnel/connectors
2026-01-09 16:57:52,985 INFO [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /opt/seatunnel/connectors
2026-01-09 16:57:52,986 INFO [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='Redis'} at: [file:/opt/seatunnel/connectors/connector-redis-2.3.12.jar]
2026-01-09 16:57:52,986 INFO [.s.p.d.AbstractPluginDiscovery] [main] - find connector jar and dependency for PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='Redis'}: [file:/opt/seatunnel/connectors/connector-redis-2.3.12.jar]
2026-01-09 16:57:52,988 INFO [p.MultipleTableJobConfigParser] [main] - start generating all sources.
2026-01-09 16:57:53,003 INFO [o.a.s.a.t.f.FactoryUtil ] [main] - get the CatalogTable from source AmazonDynamodb: .default.default.default
2026-01-09 16:57:53,008 INFO [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSource Plugin from /opt/seatunnel/connectors
2026-01-09 16:57:53,011 INFO [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='Amazondynamodb'} at: [file:/opt/seatunnel/connectors/connector-amazondynamodb-2.3.12.jar]
2026-01-09 16:57:53,011 INFO [.s.p.d.AbstractPluginDiscovery] [main] - find connector jar and dependency for PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='Amazondynamodb'}: [file:/opt/seatunnel/connectors/connector-amazondynamodb-2.3.12.jar]
2026-01-09 16:57:53,012 INFO [p.MultipleTableJobConfigParser] [main] - start generating all transforms.
2026-01-09 16:57:53,012 INFO [p.MultipleTableJobConfigParser] [main] - start generating all sinks.
2026-01-09 16:57:53,013 INFO [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /opt/seatunnel/connectors
2026-01-09 16:57:53,014 INFO [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='Redis'} at: [file:/opt/seatunnel/connectors/connector-redis-2.3.12.jar]
2026-01-09 16:57:53,014 INFO [.s.p.d.AbstractPluginDiscovery] [main] - find connector jar and dependency for PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='Redis'}: [file:/opt/seatunnel/connectors/connector-redis-2.3.12.jar]
2026-01-09 16:57:53,024 INFO [o.a.s.a.t.f.FactoryUtil ] [main] - Create sink 'Redis' with upstream input catalog-table[database: default, schema: default, table: default]
2026-01-09 16:57:53,042 INFO [o.a.s.e.c.j.ClientJobProxy ] [main] - Start submit job, job id: 1062052604328017921, with plugin jar [file:/opt/seatunnel/connectors/connector-amazondynamodb-2.3.12.jar, file:/opt/seatunnel/connectors/connector-redis-2.3.12.jar]
2026-01-09 16:57:53,047 INFO [.e.s.r.AbstractResourceManager] [hz.main.client.thread-4] - Init ResourceManager
2026-01-09 16:57:53,047 INFO [.e.s.r.AbstractResourceManager] [hz.main.client.thread-4] - initWorker...
2026-01-09 16:57:53,047 INFO [.e.s.r.AbstractResourceManager] [hz.main.client.thread-4] - init live nodes: [[localhost]:5801]
2026-01-09 16:57:53,048 INFO [.e.s.r.AbstractResourceManager] [SeaTunnel-CompletableFuture-Thread-2] - received new worker register: [localhost]:5801
2026-01-09 16:57:53,152 INFO [o.a.s.c.s.r.c.RedisParameters ] [hz.main.seaTunnel.task.thread-4] - Try to get redis version information from the jedis.info() method
2026-01-09 16:57:53,152 INFO [o.a.s.c.s.r.c.RedisParameters ] [hz.main.seaTunnel.task.thread-4] - The version of Redis is :8.4.0
2026-01-09 16:57:53,154 INFO [a.s.a.s.m.MultiTableSinkWriter] [hz.main.seaTunnel.task.thread-4] - init multi table sink writer, queue size: 1
2026-01-09 16:57:53,233 INFO [o.a.s.a.e.LoggingEventHandler ] [hz.main.generic-operation.thread-12] - log event: ReaderOpenEvent(createdTime=1767977873233, jobId=1062052604328017921, eventType=LIFECYCLE_READER_OPEN)
2026-01-09 16:57:53,402 INFO [.s.t.SourceSplitEnumeratorTask] [hz.main.seaTunnel.task.thread-4] - received reader register, readerID: TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}, taskID=1000200000000, index=0}
2026-01-09 16:57:53,429 INFO [.s.e.s.c.CheckpointCoordinator] [seatunnel-coordinator-service-9] - checkpoint is disabled, because in batch mode and 'checkpoint.interval' of env is missing.
2026-01-09 16:57:53,506 INFO [a.s.AmazonDynamoDBSourceReader] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}] - AmazonDynamoDB Source Reader [0] waiting for splits
2026-01-09 16:57:53,506 INFO [a.s.AmazonDynamoDBSourceReader] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}] - AmazonDynamoDB Source Reader [0] waiting for splits
2026-01-09 16:57:53,529 INFO [.s.t.SourceSplitEnumeratorTask] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}] - received enough reader, starting enumerator...
2026-01-09 16:57:53,530 INFO [nDynamoDBSourceSplitEnumerator] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}] - Assigning org.apache.seatunnel.connectors.seatunnel.amazondynamodb.source.AmazonDynamoDBSourceSplit@36fe5b1d to 0 reader.
2026-01-09 16:57:53,530 INFO [nDynamoDBSourceSplitEnumerator] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}] - Assigning org.apache.seatunnel.connectors.seatunnel.amazondynamodb.source.AmazonDynamoDBSourceSplit@81bd425 to 0 reader.
2026-01-09 16:57:53,530 INFO [nDynamoDBSourceSplitEnumerator] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}] - Assign splits [org.apache.seatunnel.connectors.seatunnel.amazondynamodb.source.AmazonDynamoDBSourceSplit@36fe5b1d, org.apache.seatunnel.connectors.seatunnel.amazondynamodb.source.AmazonDynamoDBSourceSplit@81bd425] to reader 0
2026-01-09 16:57:53,533 INFO [a.s.AmazonDynamoDBSourceReader] [hz.main.generic-operation.thread-21] - Reader [0] received noMoreSplit event.
2026-01-09 16:57:53,934 INFO [a.s.AmazonDynamoDBSourceReader] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}] - AmazonDynamoDB Source Reader [0] waiting for splits
2026-01-09 16:57:53,934 INFO [a.s.AmazonDynamoDBSourceReader] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}] - Closed the bounded amazonDynamodb source
2026-01-09 16:57:53,941 INFO [.s.e.s.c.CheckpointCoordinator] [hz.main.generic-operation.thread-23] - skip schedule trigger checkpoint because checkpoint type is COMPLETED_POINT_TYPE
2026-01-09 16:57:53,943 INFO [.s.e.s.c.CheckpointCoordinator] [seatunnel-coordinator-service-8] - wait checkpoint completed: 9223372036854775807
2026-01-09 16:57:55,974 INFO [.s.e.s.c.CheckpointCoordinator] [seatunnel-coordinator-service-8] - pending checkpoint(9223372036854775807/1@1062052604328017921) notify finished!
2026-01-09 16:57:55,974 INFO [.s.e.s.c.CheckpointCoordinator] [seatunnel-coordinator-service-8] - start notify checkpoint completed, job id: 1062052604328017921, pipeline id: 1, checkpoint id:9223372036854775807
2026-01-09 16:57:55,979 INFO [.s.e.s.c.CheckpointCoordinator] [seatunnel-coordinator-service-8] - start clean pending checkpoint cause CheckpointCoordinator completed.
2026-01-09 16:57:55,979 INFO [.s.e.s.c.CheckpointCoordinator] [seatunnel-coordinator-service-8] - Turn checkpoint_state_1062052604328017921_1 state from null to FINISHED
2026-01-09 16:57:55,987 INFO [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}] - [localhost]:5801 [seatunnel-420804] [5.1] taskDone, taskId = 1000100000000, taskGroup = TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}
2026-01-09 16:57:55,987 INFO [o.a.s.a.e.LoggingEventHandler ] [hz.main.generic-operation.thread-32] - log event: EnumeratorCloseEvent(createdTime=1767977875987, jobId=1062052604328017921, eventType=LIFECYCLE_ENUMERATOR_CLOSE)
2026-01-09 16:57:55,998 INFO [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}] - [localhost]:5801 [seatunnel-420804] [5.1] taskGroup TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1} complete with FINISHED
2026-01-09 16:57:55,998 INFO [o.a.s.e.s.TaskExecutionService] [hz.main.seaTunnel.task.thread-5] - [localhost]:5801 [seatunnel-420804] [5.1] Task TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1} complete with state FINISHED
2026-01-09 16:57:55,998 INFO [o.a.s.e.s.CoordinatorService ] [hz.main.seaTunnel.task.thread-5] - [localhost]:5801 [seatunnel-420804] [5.1] Received task end from execution TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}, state FINISHED
2026-01-09 16:57:56,000 INFO [o.a.s.e.s.d.p.PhysicalVertex ] [hz.main.seaTunnel.task.thread-5] - Job (1062052604328017921), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-Amazondynamodb]-SplitEnumerator (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}] turned from state RUNNING to FINISHED.
2026-01-09 16:57:56,000 INFO [o.a.s.e.s.d.p.PhysicalVertex ] [hz.main.seaTunnel.task.thread-5] - Job (1062052604328017921), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-Amazondynamodb]-SplitEnumerator (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}] state process is stopped
2026-01-09 16:57:56,000 INFO [o.a.s.e.s.d.p.SubPlan ] [seatunnel-coordinator-service-8] - Job (1062052604328017921), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-Amazondynamodb]-SplitEnumerator (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}] future complete with state FINISHED
2026-01-09 16:57:56,039 INFO [o.a.s.a.e.LoggingEventHandler ] [hz.main.generic-operation.thread-33] - log event: ReaderCloseEvent(createdTime=1767977876039, jobId=1062052604328017921, eventType=LIFECYCLE_READER_CLOSE)
2026-01-09 16:57:56,040 INFO [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}] - [localhost]:5801 [seatunnel-420804] [5.1] taskDone, taskId = 1000200000000, taskGroup = TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}
2026-01-09 16:57:56,076 INFO [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}] - [localhost]:5801 [seatunnel-420804] [5.1] taskDone, taskId = 1000200010000, taskGroup = TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}
2026-01-09 16:57:56,077 INFO [o.a.s.a.e.LoggingEventHandler ] [hz.main.generic-operation.thread-34] - log event: WriterCloseEvent(createdTime=1767977876076, jobId=1062052604328017921, eventType=LIFECYCLE_WRITER_CLOSE)
2026-01-09 16:57:56,079 INFO [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}] - [localhost]:5801 [seatunnel-420804] [5.1] taskGroup TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2} complete with FINISHED
2026-01-09 16:57:56,079 INFO [o.a.s.e.s.TaskExecutionService] [hz.main.seaTunnel.task.thread-2] - [localhost]:5801 [seatunnel-420804] [5.1] Task TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2} complete with state FINISHED
2026-01-09 16:57:56,079 INFO [o.a.s.e.s.CoordinatorService ] [hz.main.seaTunnel.task.thread-2] - [localhost]:5801 [seatunnel-420804] [5.1] Received task end from execution TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}, state FINISHED
2026-01-09 16:57:56,080 INFO [o.a.s.e.s.d.p.PhysicalVertex ] [hz.main.seaTunnel.task.thread-2] - Job (1062052604328017921), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-Amazondynamodb]-SourceTask (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}] turned from state RUNNING to FINISHED.
2026-01-09 16:57:56,080 INFO [o.a.s.e.s.d.p.PhysicalVertex ] [hz.main.seaTunnel.task.thread-2] - Job (1062052604328017921), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-Amazondynamodb]-SourceTask (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}] state process is stopped
2026-01-09 16:57:56,080 INFO [o.a.s.e.s.d.p.SubPlan ] [seatunnel-coordinator-service-8] - Job (1062052604328017921), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-Amazondynamodb]-SourceTask (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}] future complete with state FINISHED
2026-01-09 16:57:56,080 INFO [o.a.s.e.s.d.p.SubPlan ] [seatunnel-coordinator-service-8] - Job SeaTunnel_Job (1062052604328017921), Pipeline: [(1/1)] will end with state FINISHED
2026-01-09 16:57:56,081 INFO [o.a.s.e.s.m.JobMaster ] [hz.main.seaTunnel.task.thread-2] - release the task group resource TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}
2026-01-09 16:57:56,081 INFO [o.a.s.e.s.d.p.SubPlan ] [seatunnel-coordinator-service-8] - Job SeaTunnel_Job (1062052604328017921), Pipeline: [(1/1)] turned from state RUNNING to FINISHED.
2026-01-09 16:57:56,081 INFO [a.s.e.s.s.s.DefaultSlotService] [hz.main.generic-operation.thread-35] - received slot release request, jobID: 1062052604328017921, slot: SlotProfile{worker=[localhost]:5801, slotID=2, ownerJobID=1062052604328017921, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='e992c180-0d29-42db-ac3f-249a1130c0b1'}
2026-01-09 16:57:56,105 INFO [o.a.s.e.s.m.JobMaster ] [seatunnel-coordinator-service-8] - release the pipeline Job SeaTunnel_Job (1062052604328017921), Pipeline: [(1/1)] resource
2026-01-09 16:57:56,105 INFO [a.s.e.s.s.s.DefaultSlotService] [hz.main.generic-operation.thread-39] - received slot release request, jobID: 1062052604328017921, slot: SlotProfile{worker=[localhost]:5801, slotID=1, ownerJobID=1062052604328017921, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='e992c180-0d29-42db-ac3f-249a1130c0b1'}
2026-01-09 16:57:56,106 INFO [o.a.s.e.s.d.p.SubPlan ] [seatunnel-coordinator-service-8] - Job SeaTunnel_Job (1062052604328017921), Pipeline: [(1/1)] state process is stop
2026-01-09 16:57:56,106 INFO [o.a.s.e.s.d.p.PhysicalPlan ] [seatunnel-coordinator-service-7] - Job SeaTunnel_Job (1062052604328017921), Pipeline: [(1/1)] future complete with state FINISHED
2026-01-09 16:57:56,107 INFO [o.a.s.e.s.d.p.PhysicalPlan ] [seatunnel-coordinator-service-7] - Job SeaTunnel_Job (1062052604328017921) turned from state RUNNING to FINISHED.
2026-01-09 16:57:56,107 INFO [o.a.s.e.s.d.p.PhysicalPlan ] [seatunnel-coordinator-service-7] - Job SeaTunnel_Job (1062052604328017921) state process is stop
2026-01-09 16:57:56,116 INFO [o.a.s.a.e.LoggingEventHandler ] [seatunnel-coordinator-service-7] - log event: JobStateEvent(jobId=1062052604328017921, jobName=SeaTunnel_Job, jobStatus=FINISHED, createdTime=1767977876116)
2026-01-09 16:57:56,117 INFO [o.a.s.e.c.j.ClientJobProxy ] [main] - Job (1062052604328017921) end with state FINISHED
2026-01-09 16:57:56,132 INFO [s.c.s.s.c.ClientExecuteCommand] [main] -
***********************************************
Job Statistic Information
***********************************************
Start Time : 2026-01-09 16:57:52
End Time : 2026-01-09 16:57:56
Total Time(s) : 3
Total Read Count : 2
Total Write Count : 2
Total Failed Count : 0
***********************************************
2026-01-09 16:57:56,133 INFO [c.h.c.LifecycleService ] [main] - hz.client_1 [seatunnel-420804] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN
2026-01-09 16:57:56,135 INFO [c.h.i.s.t.TcpServerConnection ] [hz.main.IO.thread-in-1] - [localhost]:5801 [seatunnel-420804] [5.1] Connection[id=1, /127.0.0.1:5801->/127.0.0.1:45255, qualifier=null, endpoint=[127.0.0.1]:45255, remoteUuid=3be41dca-f0e7-4603-ad27-4e5b3aecbb4a, alive=false, connectionType=JVM, planeIndex=-1] closed. Reason: Connection closed by the other side
2026-01-09 16:57:56,135 INFO [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-420804] [5.1] Removed connection to endpoint: [localhost]:5801:579e9374-7d47-4d91-80d5-c7a9155d42d0, connection: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/127.0.0.1:45255->localhost/127.0.0.1:5801}, remoteAddress=[localhost]:5801, lastReadTime=2026-01-09 16:57:56.131, lastWriteTime=2026-01-09 16:57:56.117, closedTime=2026-01-09 16:57:56.134, connected server version=5.1}
2026-01-09 16:57:56,135 INFO [c.h.c.LifecycleService ] [main] - hz.client_1 [seatunnel-420804] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED
2026-01-09 16:57:56,136 INFO [c.h.c.i.ClientEndpointManager ] [hz.main.event-1] - [localhost]:5801 [seatunnel-420804] [5.1] Destroying ClientEndpoint{connection=Connection[id=1, /127.0.0.1:5801->/127.0.0.1:45255, qualifier=null, endpoint=[127.0.0.1]:45255, remoteUuid=3be41dca-f0e7-4603-ad27-4e5b3aecbb4a, alive=false, connectionType=JVM, planeIndex=-1], clientUuid=3be41dca-f0e7-4603-ad27-4e5b3aecbb4a, clientName=hz.client_1, authenticated=true, clientVersion=5.1, creationTime=1767977872821, latest clientAttributes=lastStatisticsCollectionTime=1767977872841,enterprise=false,clientType=JVM,clientVersion=5.1,clusterConnectionTimestamp=1767977872816,clientAddress=127.0.0.1,clientName=hz.client_1,credentials.principal=null,os.committedVirtualMemorySize=9714790400,os.freePhysicalMemorySize=2077376512,os.freeSwapSpaceSize=1073737728,os.maxFileDescriptorCount=1048576,os.openFileDescriptorCount=103,os.processCpuTime=4280000000,os.systemLoadAverage=3.26,os.totalPhysicalMemorySize=8321540096,os.totalSwapSpaceSize=1073737728,runtime.availableProcessors=14,runtime.freeMemory=286196824,runtime.maxMemory=477626368,runtime.totalMemory=376438784,runtime.uptime=2297,runtime.usedMemory=90241960, labels=[]}
2026-01-09 16:57:56,136 INFO [c.h.c.LifecycleService ] [main] - hz.client_1 [seatunnel-420804] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN
2026-01-09 16:57:56,136 INFO [s.c.s.s.c.ClientExecuteCommand] [main] - Closed SeaTunnel client......
2026-01-09 16:57:56,136 INFO [c.h.c.LifecycleService ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] [localhost]:5801 is SHUTTING_DOWN
2026-01-09 16:57:56,138 INFO [c.h.i.p.i.MigrationManager ] [hz.main.cached.thread-15] - [localhost]:5801 [seatunnel-420804] [5.1] Shutdown request of Member [localhost]:5801 - 579e9374-7d47-4d91-80d5-c7a9155d42d0 [master node] [active master] this is handled
2026-01-09 16:57:56,141 INFO [c.h.i.i.Node ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Shutting down connection manager...
2026-01-09 16:57:56,142 INFO [c.h.i.i.Node ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Shutting down node engine...
2026-01-09 16:57:56,149 INFO [.s.s.o.e.j.s.AbstractConnector] [main] - Stopped ServerConnector@6e31d989{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2026-01-09 16:57:56,149 INFO [o.a.s.s.o.e.j.s.session ] [main] - node0 Stopped scavenging
2026-01-09 16:57:56,150 INFO [a.s.s.o.e.j.s.h.ContextHandler] [main] - Stopped o.a.s.s.o.e.j.s.ServletContextHandler@7069f076{/,null,STOPPED}
2026-01-09 16:57:56,151 INFO [.c.c.DefaultClassLoaderService] [main] - close classloader service
2026-01-09 16:57:56,152 INFO [o.a.s.e.s.EventService ] [event-forwarder-0] - Event forward thread interrupted
2026-01-09 16:57:56,155 INFO [c.h.i.i.NodeExtension ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Destroying node NodeExtension.
2026-01-09 16:57:56,156 INFO [c.h.i.i.Node ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Hazelcast Shutdown is completed in 18 ms.
2026-01-09 16:57:56,156 INFO [c.h.c.LifecycleService ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] [localhost]:5801 is SHUTDOWN
2026-01-09 16:57:56,156 INFO [s.c.s.s.c.ClientExecuteCommand] [main] - Closed HazelcastInstance ......
2026-01-09 16:57:56,156 INFO [s.c.s.s.c.ClientExecuteCommand] [main] - Closed metrics executor service ......
2026-01-09 16:57:56,156 INFO [s.c.s.s.c.ClientExecuteCommand] [SeaTunnel-CompletableFuture-Thread-7] - run shutdown hook because get close signal任务执行完成后，用于执行它的 Docker 容器会被终止，此时你就能在 Redis 中看到写入的数据记录了。
正如 SeaTunnel 官方所说，整个过程简单到只需写一个配置文件。当然，要真正吃透 source 和 sink 连接器提供的众多配置项，可能还需要花些时间。你可以点击此处查看 Amazon DynamoDB 的配置属性，点击此处查看 Redis 的配置属性。
方案二：生产级引擎（集群模式）
当你需要一套常驻在线的数据管道基础设施，能够处理多个任务、按需扩容，并提供 REST API 访问时，集群模式就是你的选择。
第一步：搭建基础设施
创建一个 docker-compose.yaml，拉起一个完整的 SeaTunnel 集群。它同时也会带上 Redis------在这个用例里，Redis 是数据管道的目标端（sink）。
version: '3.8'
services:
seatunnel-master:
image: apache/seatunnel:2.3.12
container_name: seatunnel-master
environment:
- ST_DOCKER_MEMBER_LIST=172.16.0.3,172.16.0.4,172.16.0.5
volumes:
- ./config/hazelcast-master.yaml:/opt/seatunnel/config/hazelcast-master.yaml
- ./config/seatunnel.yaml:/opt/seatunnel/config/seatunnel.yaml
entrypoint: >
/bin/sh -c "
/opt/seatunnel/bin/seatunnel-cluster.sh -r master
"
ports:
- "5801:5801"
- "8080:8080" # REST API 端点
networks:
seatunnel_network:
ipv4_address: 172.16.0.3
healthcheck:
test: [ "CMD-SHELL", "curl -f http://localhost:8080/system-monitoring-information || exit 1" ]
interval: 30s
timeout: 10s
retries: 10
start_period: 60s
seatunnel-worker1:
image: apache/seatunnel:2.3.12
container_name: seatunnel-worker1
environment:
- ST_DOCKER_MEMBER_LIST=172.16.0.3,172.16.0.4,172.16.0.5
volumes:
- ./config/hazelcast-worker.yaml:/opt/seatunnel/config/hazelcast-worker.yaml
- ./config/seatunnel.yaml:/opt/seatunnel/config/seatunnel.yaml
entrypoint: >
/bin/sh -c "
/opt/seatunnel/bin/seatunnel-cluster.sh -r worker
"
depends_on:
- seatunnel-master
networks:
seatunnel_network:
ipv4_address: 172.16.0.4
healthcheck:
test: ["CMD-SHELL", "pgrep -f seatunnel || exit 1"]
interval: 30s
timeout: 10s
retries: 5
start_period: 45s
seatunnel-worker2:
image: apache/seatunnel:2.3.12
container_name: seatunnel-worker2
environment:
- ST_DOCKER_MEMBER_LIST=172.16.0.3,172.16.0.4,172.16.0.5
volumes:
- ./config/hazelcast-worker.yaml:/opt/seatunnel/config/hazelcast-worker.yaml
- ./config/seatunnel.yaml:/opt/seatunnel/config/seatunnel.yaml
entrypoint: >
/bin/sh -c "
/opt/seatunnel/bin/seatunnel-cluster.sh -r worker
"
depends_on:
- seatunnel-master
networks:
seatunnel_network:
ipv4_address: 172.16.0.5
healthcheck:
test: ["CMD-SHELL", "pgrep -f seatunnel || exit 1"]
interval: 30s
timeout: 10s
retries: 5
start_period: 45s
redis-database:
container_name: redis-database
hostname: redis-database
image: redis:8.4.0
ports:
- "6379:6379"
networks:
seatunnel_network:
ipv4_address: 172.16.0.2
healthcheck:
test: [ "CMD-SHELL", "redis-cli ping | grep PONG" ]
interval: 10s
retries: 5
start_period: 5s
timeout: 5s
networks:
seatunnel_network:
driver: bridge
ipam:
config:
- subnet: 172.16.0.0/24第二步：配置集群
创建 config/seatunnel.yaml 以启用 REST API：
seatunnel:
engine:
http:
enable-http: true
port: 8080创建 config/hazelcast-master.yaml 和 config/hazelcast-worker.yaml：
hazelcast:
network:
rest-api:
enabled: true
endpoint-groups:
CLUSTER_READ:
enabled: true
CLUSTER_WRITE:
enabled: true
HEALTH_CHECK:
enabled: true
DATA:
enabled: true
port:
auto-increment: true
port-count: 100
port: 5801
join:
tcp-ip:
enabled: true
member-list:
- 172.16.0.3
- 172.16.0.4
- 172.16.0.5第三步：启动集群
docker compose up -d现在你的 SeaTunnel 集群已运行，包含：
- 1 个 Master 节点（协调器 + API 服务器）
- 2 个 Worker 节点（实际执行数据管道）
- Redis 数据库（你的 sink 和目标系统）
第四步：通过 REST API 提交任务
接下来是见证奇迹的时刻！用一条简单的 HTTP POST 提交任务：
curl -X POST http://localhost:8080/submit-job?jobName=dynamodb-to-redis \
-H "Content-Type: application/json" \
-d '{
"env": {
"parallelism": 2,
"job.mode": "BATCH"
},
"source": [
{
"plugin_name": "Amazondynamodb",
"plugin_output": "dynamodb_source",
"url": "http://dynamodb.us-east-1.amazonaws.com",
"region": "us-east-1",
"access_key_id": "YOUR_ACCESS_KEY",
"secret_access_key": "YOUR_SECRET_KEY",
"table": "mySourceTable",
"schema": {
"fields": {
"customerId": "int",
"customerName": "string",
"address": "string"
}
}
}
],
"transform": [],
"sink": [
{
"plugin_name": "Redis",
"host": "redis-database",
"port": 6379,
"support_custom_key": true,
"key": "customer:{customerId}",
"data_type": "hash"
}
]
}'任务一旦提交，就会立刻开始执行。你可以在 SeaTunnel 提供的 Web 控制台里跟进任务执行情况：访问 http://localhost:8080/#/overview，然后切换到 Jobs 标签页。
总结
有了 Apache SeaTunnel，把数据从 Amazon DynamoDB 同步到 Redis 就变成了一件"配置"层面的工作。无论你需要一次性迁移，还是一套常驻在线的数据集成管道，Apache SeaTunnel 都有满足你需求的工具。
使用 SeaTunnel 的妙处在于：
- 无需维护代码------只需简单配置即可使用
- 久经实战检验的连接器------专注于你的业务逻辑
- 部署灵活------无论是笔记本还是生产集群都能跑
- 天生可扩展------可随数据增长横向扩容
准备好试一试了吗？建议先用单机模式快速实验一下，等要上生产时再升级到集群模式。