一、背景与目的
测试与验证HA模式数据全量迁移到分片集群,
工具: redisShake4.3.1
二、准备
-
原集群
主: 172.25.242.158
从: 172.25.242.154免密
SOX主机
端口:6379
redis-cli -h 172.25.242.158 -n 1 dbsize -
目的集群
makefile
#
172.25.39.54:6379
172.25.39.51:6379
172.25.39.52:6379
172.25.39.48:6379
172.25.39.55:6379
172.25.39.49:6379
172.25.39.53:6379
172.25.39.50:6379
组名: redis-cluster-newlps-ksa
密码: 2ghlmvl_ST
端口:6379
#
redis-cli -h 172.25.39.48 -a 2ghlmvl_MT cluster nodes
# 测试分片集群
172.25.190.13
172.25.189.209
172.25.189.211
172.25.190.12
172.25.190.11
172.25.189.212
密码
123456
redis-cli -h 172.25.190.13 -a 123456 dbsize
# redis-cli -h 172.25.189.209 -a 123456 dbsize
redis-cli -h 172.25.189.211 -a 123456 dbsize
redis-cli -h 172.25.190.12 -a 123456 dbsize
redis-cli -h 172.25.190.11 -a 123456 dbsize
redis-cli -h 172.25.189.212 -a 123456 dbsize
- 工具
bash
/root/redis/redisshake431
三、全量数据同步源
原集群数据量检查
bash
[root@KSSYYVI01352 redisshake431]# redis-cli -h 172.25.242.158 -n 1 dbsize
(integer) 106992
[root@KSSYYVI01352 redisshake431]# redis-cli -h 172.25.242.158 -n 10 dbsize
(integer) 2486
-
redisShake配置文件调整shake.toml
ini
[sync_reader]
cluster = false
address = "172.25.252.154:6379" # 源 Redis 地址
password = "" # 源 Redis 密码
sync_rdb = false # 启用 RDB 同步
sync_aof = false # 关闭 AOF 同步
[redis_writer]
cluster = true
address = "172.25.190.13:6379" # 目标 Redis 地址
password = "123456" # 目标 Redis 密码
skip_existing_keys = true
[filter]
allow_db = [01,10] # 只同步 1和10 号库的数据
#allow_keys = ["ycms:sm:draw:lottery:prize:current:count:2e2f3471006540194639:total"]
[advanced]
log_file = "shake.log" # 日志文件路径
log_level = "info" # 日志级别:info
pipeline_count_limit = 1024 # 默认管道大小
#rdb_restore_command_behavior = "skip" # 遇到重复键时覆盖
-
执行
bash
#
./redis-shake shake.toml
#
tail -f shake.log
shake.log 中显示如下:

执行的输出:
yaml
[root@KSSYYVI01352 redisshake431]# ./redis-shake shake.toml
2025-04-17 19:52:07 INF load config from file: shake.toml
2025-04-17 19:52:07 INF log_level: [info], log_file: [/root/redis/redisshake431/data/shake.log]
2025-04-17 19:52:07 INF changed work dir. dir=[/root/redis/redisshake431/data]
2025-04-17 19:52:07 INF GOMAXPROCS defaults to the value of runtime.NumCPU [4]
2025-04-17 19:52:07 INF not set pprof port
2025-04-17 19:52:07 INF create SyncStandaloneReader
2025-04-17 19:52:07 INF * address: 172.25.242.158:6379
2025-04-17 19:52:07 INF * username:
2025-04-17 19:52:07 INF * password:
2025-04-17 19:52:07 INF * tls: false
2025-04-17 19:52:07 INF create RedisClusterWriter
2025-04-17 19:52:07 INF * address (should be the address of one node in the Redis cluster): 172.25.190.13:6379
2025-04-17 19:52:07 INF * username:
2025-04-17 19:52:07 INF * password: ******
2025-04-17 19:52:07 INF * tls: false
2025-04-17 19:52:07 INF address=172.25.190.13:6379, reply=f99fe6a48492bd388474f33771a18b04fbf3c3d5 172.25.190.13:6379@16379 myself,master - 0 1744890726000 8 connected 5462-10922
c5815dabf9896b15ceda8dbd15742ab15d3e9fe6 172.25.190.11:6379@16379 slave 82a78b1cb3aaf11343704fb39f32e28f702fbfcc 0 1744890725039 6 connected
2150fe1f0122b2b03f2ab75ccd9516288800a2c8 172.25.189.211:6379@16379 master - 0 1744890727042 9 connected 10923-16383
82a78b1cb3aaf11343704fb39f32e28f702fbfcc 172.25.189.212:6379@16379 master - 0 1744890726039 6 connected 0-5461
5ec837e77676d921e135f7879c044786f7754b30 172.25.190.12:6379@16379 slave f99fe6a48492bd388474f33771a18b04fbf3c3d5 0 1744890725000 8 connected
e49b1fe2d24c3cd4452b32c9d78f99a2e9300c37 172.25.189.209:6379@16379 slave 2150fe1f0122b2b03f2ab75ccd9516288800a2c8 0 1744890725000 9 connected
2025-04-17 19:52:07 INF redisClusterWriter connected to redis cluster successful. addresses=[172.25.190.13:6379 172.25.189.211:6379 172.25.189.212:6379]
2025-04-17 19:52:07 INF start syncing...
2025-04-17 19:52:07 INF [reader_172.25.242.158_6379] source db is not doing bgsave! continue.
2025-04-17 19:52:11 ERR [writer_172.25.189.211_6379] receive reply failed. cmd=[select 1], error=[ERR SELECT is not allowed in cluster mode]
RedisShake/internal/writer/redis_standalone_writer.go:154 -> (*redisStandaloneWriter).processReply()
runtime/asm_amd64.s:1650 -> goexit()

说明:
主从模式向cluster迁移数据的时候,由于cluster模式不支持切换db,导致迁移失败。
cluster不支持select 选择db, 而源redis集群数据并不在db01中,所以失败。
建议:
如果源 Redis 使用了特定的数据库编号(如 1
或 10
),而目标 Redis 集群不支持数据库编号,那么需要在源 Redis 中将数据迁移到默认数据库(通常是数据库 0
)。
观察
css
#
redis-cli -h 172.25.190.13 -a 123456 dbsize
redis-cli -h 172.25.189.209 -a 123456 dbsize
redis-cli -h 172.25.189.211 -a 123456 dbsize
redis-cli -h 172.25.190.12 -a 123456 dbsize
redis-cli -h 172.25.190.11 -a 123456 dbsize
redis-cli -h 172.25.189.212 -a 123456 dbsize
对比新旧集群的数据量
css
# 新集群
redis-cli -h 172.25.39.54 -p 6379 -a 2ghlmvl_MT dbsize
redis-cli -h 172.25.39.51 -p 6379 -a 2ghlmvl_MT dbsize
redis-cli -h 172.25.39.52 -p 6379 -a 2ghlmvl_MT dbsize
redis-cli -h 172.25.39.48 -p 6379 -a 2ghlmvl_MT dbsize
redis-cli -h 172.25.39.55 -p 6379 -a 2ghlmvl_MT dbsize
redis-cli -h 172.25.39.49 -p 6379 -a 2ghlmvl_MT dbsize
redis-cli -h 172.25.39.53 -p 6379 -a 2ghlmvl_MT dbsize
redis-cli -h 172.25.39.50 -p 6379 -a 2ghlmvl_MT dbsize
完成
四、补充信息
- 文档与参考
tair-opensource.github.io/RedisShake/...
- 相关方案 lue