企业级NoSql数据库Redis集群

1.关系型数据库和NoSQL数据库

1.1 数据库主要分为两大类：关系型数据库与 NoSQL 数据库

关系型数据库

是建立在关系模型基础上的数据库，其借助于集合代数等数学概念和方法来处理数据库中的数据主流的 MySQL、Oracle、MS SQL Server 和 DB2 都属于这类传统数据库。

NoSQL 数据库

全称为 Not Only SQL，意思就是适用关系型数据库的时候就使用关系型数据库，不适用的时候也没有必要非使用关系型数据库不可，可以考虑使用更加合适的数据存储。主要分为临时性键值存储（memcached、Redis）、永久性键值存储（ROMA、Redis）、面向文档的数据库（MongoDB、CouchDB）、面向列的数据库（Cassandra、HBase），每种 NoSQL 都有其特有的使用场景及优点。

1.2 为什么还要用 NoSQL 数据库呢？

主要是由于随着互联网发展，数据量越来越大，对性能要求越来越高，传统数据库存在着先天性的缺陷，即单机（单库）性能瓶颈，并且扩展困难。这样既有单机单库瓶颈，却又扩展困难，自然无法满足日益增长的海量数据存储及其性能要求，所以才会出现了各种不同的 NoSQL 产品，NoSQL 根本性的优势在于在云计算时代，简单、易于大规模分布式扩展，并且读写性能非常高

2.Remote Dictionary Server 简介

2.1 redis特性

速度快: 10W QPS,基于内存,C语言实现

单线程

持久化

支持多种数据结构

支持多种编程语言

功能丰富: 支持Lua脚本,发布订阅,事务,pipeline等功能

简单: 代码短小精悍(单机核心代码只有23000行左右),单线程开发容易,不依赖外部库,使用简单

主从复制

支持高可用和分布式

单线程为何如此快?

纯内存

非阻塞

避免线程切换和竞态消耗

2.2 Redis应用场景

Session 共享：常见于web集群中的Tomcat或者PHP中多web服务器session共享

缓存：数据查询、电商网站商品信息、新闻内容

计数器：访问排行榜、商品浏览数等和次数相关的数值统计场景

微博/微信社交场合：共同好友,粉丝数,关注,点赞评论等

消息队列：ELK的日志缓存、部分业务的订阅发布系统

地理位置: 基于GEO(地理信息定位),实现摇一摇,附近的人,外卖等功能

2.3缓存的实现流程

数据更新操作流程

数据读操作流程

3.redis的安装

3.1rpm包安装

[root@redis-node1 ~]# dnf install redis -y

3.2源码安装

#####解压
[root@redis-node1 ~]# tar zxf redis-7.4.0.tar.gz
[root@redis-node1 ~]# ls
redis-7.4.0 redis-7.4.0.tar.gz

###安装编译工具######所有要装redis的主机都要装
[root@redis-node1 redis-7.4.0]# dnf install make gcc initscripts-10.11.6-1.el9.x86_64 -y

#执行编译命令
[root@redis-node1 redis-7.4.0]# make
[root@redis-node1 redis-7.4.0]# make install
#启动Redis
[root@redis-node1 redis-7.4.0]# cd utils/
[root@redis-node1 utils]# ./install_server.sh
Welcome to the redis service installer
This script will help you easily set up a running redis server
This systems seems to use systemd. #提示系统使用的是systemd的初始化方式
Please take a look at the provided example service unit files in this directory, 
and adapt and install them. Sorry!

[root@redis-node1 utils]# vim install_server.sh #解决报错问题
#bail if this system is managed by systemd
#_pid_1_exe="$(readlink -f /proc/1/exe)"
#if [ "${_pid_1_exe##*/}" = systemd ]
#then
#       echo "This systems seems to use systemd."
#       echo "Please take a look at the provided example service unit files in 
this directory, and adapt and install them. Sorry!"
#       exit 1
#fi

###初始化
[root@redis-node1 utils]# ./install_server.sh

#配置redis
[root@redis-node1 utils]# vim /etc/redis/6379.conf
bind * -::*
protected-mode no

[root@redis-node1 utils]# /etc/init.d/redis_6379 restart
Stopping ...
Redis stopped
Starting Redis server...
[root@redis-node1 utils]# netstat -antlpe | grep redis
tcp        0      0 0.0.0.0:6379            0.0.0.0:*               LISTEN      0 
         67267      38417/redis-server
tcp6       0      0 :::6379

拷贝文件到其他主机

172.25.250.20/172.25.250.30

[root@redis-node1 ~]# ls
anaconda-ks.cfg  redis-7.4.0  redis-7.4.0.tar.gz
[root@redis-node1 ~]# scp -r redis-7.4.0 root@172.25.250.20:/root

[root@redis-node1 ~]# cd /usr/local/bin/
[root@redis-node1 bin]# ls
redis-benchmark  redis-check-aof  redis-check-rdb  redis-cli  redis-sentinel  redis-server
[root@redis-node1 bin]# ll
total 29416
-rwxr-xr-x 1 root root  6365824 Aug 25 21:57 redis-benchmark
lrwxrwxrwx 1 root root       12 Aug 25 21:57 redis-check-aof -> redis-server
lrwxrwxrwx 1 root root       12 Aug 25 21:57 redis-check-rdb -> redis-server
-rwxr-xr-x 1 root root  7207664 Aug 25 21:57 redis-cli
lrwxrwxrwx 1 root root       12 Aug 25 21:57 redis-sentinel -> redis-server
-rwxr-xr-x 1 root root 16540544 Aug 25 21:57 redis-server
[root@redis-node1 bin]# rsync -al * root@172.25.250.20:/usr/local/bin


###到172.25.250.20查看
[root@redis-node2 ~]# ls -l /usr/local/bin/
total 29416
-rwxr-xr-x 1 root root  6365824 Aug 25 21:57 redis-benchmark
lrwxrwxrwx 1 root root       12 Aug 25 21:57 redis-check-aof -> redis-server
lrwxrwxrwx 1 root root       12 Aug 25 21:57 redis-check-rdb -> redis-server
-rwxr-xr-x 1 root root  7207664 Aug 25 21:57 redis-cli
lrwxrwxrwx 1 root root       12 Aug 25 21:57 redis-sentinel -> redis-server
-rwxr-xr-x 1 root root 16540544 Aug 25 21:57 redis-server

[root@redis-node2 ~]# cd redis-7.4.0/
[root@redis-node2 redis-7.4.0]# ls
00-RELEASENOTES     INSTALL      redis.conf              runtest-sentinel  TLS.md
BUGS                LICENSE.txt  REDISCONTRIBUTIONS.txt  SECURITY.md       utils
CODE_OF_CONDUCT.md  Makefile     runtest                 sentinel.conf
CONTRIBUTING.md     MANIFESTO    runtest-cluster         src
deps                README.md    runtest-moduleapi       tests
[root@redis-node2 redis-7.4.0]# cd utils/
[root@redis-node2 utils]# ls
build-static-symbols.tcl    graphs                 reply_schema_linter.js
cluster_fail_time.tcl       hyperloglog            req-res-log-validator.py
corrupt_rdb.c               install_server.sh      req-res-validator
create-cluster              lru                    speed-regression.tcl
generate-command-code.py    redis-copy.rb          srandmember
generate-commands-json.py   redis_init_script      systemd-redis_multiple_servers@.service
generate-fmtargs.py         redis_init_script.tpl  systemd-redis_server.service
generate-module-api-doc.rb  redis-sha1.rb          tracking_collisions.c
gen-test-certs.sh           releasetools           whatisdoing.sh

####初始化
[root@redis-node2 utils]# ./install_server.sh 

###改配置文件
[root@redis-node2 utils]# vim /etc/redis/6379.conf 
bind * -::*
protected-mode no

[root@redis-node2 ~]# /etc/init.d/redis_6379 restart

4.redis基本操作

示例

#查看配置
127.0.0.1:6379[1]> CONFIG GET bind
1) "bind"
2) "* -::*"
127.0.0.1:6379[1]> CONFIG GET * 

#写入和读取数据
127.0.0.1:6379> SET name lee
OK
127.0.0.1:6379> GET name
"lee"
127.0.0.1:6379> set name lee ex 5
OK
127.0.0.1:6379> get name
"lee"
127.0.0.1:6379> get name
"lee"
127.0.0.1:6379> get name
"lee"
127.0.0.1:6379> get name
"lee"
127.0.0.1:6379> get name
(nil)
#如果没有设定数据过期时间会一直存在， /var/lib/redis/6379/dump.rdb内存快照中
127.0.0.1:6379> set name lee
OK
127.0.0.1:6379> KEYS * #查看所有key

#选择数据库 redisa中有0-15个数据库
127.0.0.1:6379> select 1
OK
127.0.0.1:6379[1]> get name
(nil)
127.0.0.1:6379> select 0
127.0.0.1:6379[1]> select 16
(error) ERR DB index is out of range

#移动数据
127.0.0.1:6379> set name lee
OK
127.0.0.1:6379> MOVE name 1
(integer) 1
127.0.0.1:6379> GET name
(nil)
127.0.0.1:6379> select 1
OK
127.0.0.1:6379[1]> get name
"lee"

#改变键名
127.0.0.1:6379[1]> RENAME name id
OK
127.0.0.1:6379[1]> get name
(nil)
127.0.0.1:6379[1]> get id
"lee"

#设定数据过期时间
127.0.0.1:6379> set name lee ex 10000
OK
127.0.0.1:6379> get name
"lee"
127.0.0.1:6379> EXPIRE name 3
(integer) 1
127.0.0.1:6379> get name
"lee"
127.0.0.1:6379> get name
(nil)

#删除
127.0.0.1:6379> set name lee
OK
127.0.0.1:6379> get name
"lee"
127.0.0.1:6379> del name
(integer) 1
127.0.0.1:6379> get name
(nil)


#持久化保存
127.0.0.1:6379> PERSIST name
(integer) 0


#判断key是否存在
127.0.0.1:6379> EXISTS name
(integer) 1
127.0.0.1:6379> EXISTS lee
(integer) 0 

#清空当前库
127.0.0.1:6379> flushdb
OK
127.0.0.1:6379> GET name
(nil)

#清空所有库
127.0.0.1:6379[1]> FLUSHALL
OK

5.2配置主从同步

修改master节点配置文件

[root@redis-node1 & node2 & node3 ~]# vim /etc/redis/6379.confprotected-mode no           #关闭protected模式
[root@redis-node1 &node2 & node3 ~]# /etc/init.d/redis_6379 restart

测试效果

######在master节点
[root@redis-node1 ~]# redis-cli 
127.0.0.1:6379> set name ren
OK
127.0.0.1:6379> get name
"ren"
####在slave1查看
[root@redis-node2 ~]# redis-cli 
127.0.0.1:6379> get name
"ren"
####在slave2查看
[root@redis-node3 utils]# redis-cli 
127.0.0.1:6379> get name
"ren"

5.3主从同步过程

slave节点发送请求到master节点
slave节点通过master节点的认证开始进行同步
master节点会开启bgsave进程发送内存到rbd到slave节点，在此过程中是异步操作，也就是master节点仍然可以进行写入操作
slave节点收到rbd后首先清空自己的所有数据
slave节点加载rdb并进行数据恢复
在master和slave同步过程中master还会开启新的bgsave进程把没有同步的数据进行缓存
然后通过自有的replactionfeeslave函数把未通过的内存快照发送到slave的数据一条一条写入到slave中

6.redis的哨兵（高可用）

实验环境：我们用一主两从来实现redis的高可用架构

6.1 redis哨兵

sentinel进程是用于监控redis集群中master主服务器工作的状态，在master主服务器发送故障时，可以实现master和slave的切换，保证系统高可用
每个哨兵（sentinel）进程会向其他哨兵、master、slave定时发送信息，以确认对方是否"存活"，如果发现对方在指定的配置时间（此项可配置）内未得到回应，则暂时认定对方离线，也就是所谓的"主机认为宕机"----主观认定

SDOWN
有主观宕机也有客观宕机。当哨兵群中多数哨兵（sentinel）进程在对master主服务器做出主观宕机（SDOWN）的判读，并且通过SENTINEL is-master-down-by-addr命令互相交流之后，得出master server下线判断，这就是客观宕机 ------ODOWN
通过一定的vote算法，从剩下的slave服务器的节点中，选一台提升为master，然后自动修改相关配置文件，并开启故障转移
sentinel机制可以解决master和slave之间的自动切换问题，当单个master的性能瓶颈问题无法出解决，类似于mysql中的MHA功能
Redis Sentinel中的Sentinel节点个数应该为大于等于3且最好为奇数

sentinel中的三个定时任务

每10s每个sentinel对master和slave执行info
- 发现slave节点
- 确认主从关系
每2s每个sentinel通过master节点的channel交换信息（pub/sub）
- 通过sentinel_:hello频道交互
- 交互对节点的看法和自身信息
每1s每个sentinel对其他sentinel和redis执行ping

6.2 哨兵的实验过程

在所有阶段中关闭 protected-mode no

在master节点中

[root@redis-node1 ~]# cd redis-7.4.0/
[root@redis-node1 redis-7.4.0]# ls
00-RELEASENOTES     INSTALL      redis.conf              runtest-sentinel  TLS.md
BUGS                LICENSE.txt  REDISCONTRIBUTIONS.txt  SECURITY.md       utils
CODE_OF_CONDUCT.md  Makefile     runtest                 sentinel.conf
CONTRIBUTING.md     MANIFESTO    runtest-cluster         src
deps                README.md    runtest-moduleapi       tests

[root@redis-node1 redis-7.4.0]# vim sentinel.conf 
protected-mode no
sentinel monitor mymaster 172.25.250.10 6379 2
sentinel down-after-milliseconds mymaster 10000

[root@redis-node1 redis-7.4.0]# scp /etc/redis/sentinel.conf root@172.25.250.20:/etc/redis/sentinel.conf
root@172.25.250.20's password: 
sentinel.conf                                             100%   14KB  12.3MB/s   00:00    
[root@redis-node1 redis-7.4.0]# scp /etc/redis/sentinel.conf root@172.25.250.30:/etc/redis/sentinel.conf
root@172.25.250.30's password: 
sentinel.conf                                             100%   14KB  15.3MB/s   00:00    
[root@redis-node1 redis-7.4.0]# cd /etc/redis/
[root@redis-node1 redis]# ls
6379.conf  sentinel.conf

#########因为当master坏掉又修复好了之后，他会成为slave，然后他会还原数据或者配置文件，所以需要备份
####所有主机都操作
[root@redis-node1 redis]# cp sentinel.conf sentinel.conf.bak

启动服务

####172.25.250.10（master）开启
[root@redis-node1 redis-7.4.0]# redis-sentinel /etc/redis/sentinel.conf
8281:X 26 Aug 2024 13:52:15.949 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
8281:X 26 Aug 2024 13:52:15.949 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
8281:X 26 Aug 2024 13:52:15.949 * Redis version=7.4.0, bits=64, commit=00000000, modified=0, pid=8281, just started
8281:X 26 Aug 2024 13:52:15.949 * Configuration loaded
8281:X 26 Aug 2024 13:52:15.949 * Increased maximum number of open files to 10032 (it was originally set to 1024).
8281:X 26 Aug 2024 13:52:15.949 * monotonic clock: POSIX clock_gettime
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis Community Edition      
  .-`` .-```.  ```\/    _.,_ ''-._     7.4.0 (00000000/0) 64 bit
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 26379
 |    `-._   `._    /     _.-'    |     PID: 8281
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           https://redis.io       
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

8281:X 26 Aug 2024 13:52:15.951 * Sentinel new configuration saved on disk
8281:X 26 Aug 2024 13:52:15.951 * Sentinel ID is 1823ba60190a86e7653368c718b5b733b70ead80
8281:X 26 Aug 2024 13:52:15.951 # +monitor master mymaster 172.25.250.10 6379 quorum 2
8281:X 26 Aug 2024 13:52:15.952 * +slave slave 172.25.250.20:6379 172.25.250.20 6379 @ mymaster 172.25.250.10 6379
8281:X 26 Aug 2024 13:52:15.954 * Sentinel new configuration saved on disk
8281:X 26 Aug 2024 13:52:15.954 * +slave slave 172.25.250.30:6379 172.25.250.30 6379 @ mymaster 172.25.250.10 6379
8281:X 26 Aug 2024 13:52:15.955 * Sentinel new configuration saved on disk
8281:X 26 Aug 2024 13:52:30.651 * +sentinel sentinel a23f818e9bd026df515af54110dc6d393338eb60 172.25.250.20 26379 @ mymaster 172.25.250.10 6379
8281:X 26 Aug 2024 13:52:30.653 * Sentinel new configuration saved on disk
####其他一样操作
[root@redis-node1 ~]# redis-cli 
127.0.0.1:6379> SHUTDOWN
(1.23s)

###然后我们关闭master模拟故障

####去slave上查看，可以看到node2是master，slave只有172.25.250.30（node3），因为旧master关闭了，
[root@redis-node2 redis]# redis-cli 
127.0.0.1:6379> info replication
# Replication
role:master            #####现在node2是master
connected_slaves:1
slave0:ip=172.25.250.30,port=6379,state=online,offset=23791,lag=0
master_failover_state:no-failover
master_replid:3737c2502945b0e05f08a2eedfa23a4405f670a3
master_replid2:b4d9ee5140c8c2045ca62f042f68f2ec27054c5e
master_repl_offset:23791
second_repl_offset:11700
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:23791

####然后我们开启172.25.250.10并查看他的状态，发现他已经变为slave
[root@redis-node1 ~]# /etc/init.d/redis_6379 start
Starting Redis server...
[root@redis-node1 ~]# redis-cli 
127.0.0.1:6379> info replication
# Replication
role:slave       ##############变为slave
master_host:172.25.250.20       ####指向172.25.250.20
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_read_repl_offset:37871
slave_repl_offset:37871
slave_priority:100
slave_read_only:1
replica_announced:1
connected_slaves:0
master_failover_state:no-failover
master_replid:3737c2502945b0e05f08a2eedfa23a4405f670a3
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:37871
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:34578
repl_backlog_histlen:3294

####172.25.250,30查看
[root@redis-node3 redis]# redis-cli 
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:172.25.250.20        #######指向20
master_port:6379
master_link_status:up
master_last_io_seconds_ago:9
master_sync_in_progress:0
slave_read_repl_offset:125665
slave_repl_offset:125665
slave_priority:100
slave_read_only:1
replica_announced:1
connected_slaves:0
master_failover_state:no-failover
master_replid:3737c2502945b0e05f08a2eedfa23a4405f670a3
master_replid2:b4d9ee5140c8c2045ca62f042f68f2ec27054c5e
master_repl_offset:125665
second_repl_offset:11700
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:224
repl_backlog_histlen:125442


####我们再去172.25.250.20（新master）查看，会发现172.25.250.10已经上线，并且是slave状态
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=172.25.250.30,port=6379,state=online,offset=33731,lag=0
slave1:ip=172.25.250.10,port=6379,state=wait_bgsave,offset=0,lag=0
master_failover_state:no-failover
master_replid:3737c2502945b0e05f08a2eedfa23a4405f670a3
master_replid2:b4d9ee5140c8c2045ca62f042f68f2ec27054c5e
master_repl_offset:33872
second_repl_offset:11700
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:33872


###注意
/etc/redis/sentinel.conf 文件在用哨兵程序调用后会更改其配置文件，如果需要重新做需要删掉文件重新编辑

监控172.25.250,10状态

监控172.25.250.20状态

监控172.25.250.30状态

6.3 在整个架构中可能会出现的问题

问题：

在生产环境中如果master和slave中的网络出现故障，由于哨兵的存在会把master提出去，当网络恢复后，master发现环境发生改变，master会把自己的身份变成slave

master变成slave后会把网络故障那段时间写入自己中的数据清掉，这样数据就丢失了

解决：

master在被写入数据时会持续链接slave，master确保有两个slave可以写入我才允许写入，如果slave数量少于2个便拒绝写入

在master中设定

现在的master是172.25.250.20

[root@redis-node2 redis]# redis-cli 
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=172.25.250.30,port=6379,state=online,offset=127149,lag=0
slave1:ip=172.25.250.10,port=6379,state=online,offset=127149,lag=0
master_failover_state:no-failover
master_replid:3737c2502945b0e05f08a2eedfa23a4405f670a3
master_replid2:b4d9ee5140c8c2045ca62f042f68f2ec27054c5e
master_repl_offset:127149
second_repl_offset:11700
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:127149

#########这时0个slave也可以写入
127.0.0.1:6379> CONFIG GET min-slaves-to-write
1) "min-slaves-to-write"
2) "0"

########设置必须最少2个master才能写入
127.0.0.1:6379> CONFIG set min-slaves-to-write 2
OK
127.0.0.1:6379> CONFIG GET min-slaves-to-write
1) "min-slaves-to-write"
2) "2"
127.0.0.1:6379> 

#####以上是临时设置

#########如果要永久设置
min-slaves-to-write 2

7.redis cluster（无中心化设计）

7.1 redis cluster工作原理

在哨兵sentinel机制中，可以解决redis高可用问题，即当master故障可以自动将slave提升为master，从而可以保证redis服务正常使用，但是无法解决redis单机写入的瓶颈问题，即单机redis写入性能限制于单机内存大小，并发数量，网卡速率等因素'

redis cluster特点

所有redis节点使用（ping机制）互联
集群中某个节点是否失效，由整个集群值超过半数的节点监测都失效。才算真正的失效
客户端不需要proxy即可直接连接redis，应用程序中需要配置有全部的redis服务器ip
redis cluster把所有的redis node 平均映射到 0-16383个槽位(slot)上，读写需要到指定的redis node上进行操作，因此有多少个redis node相当于redis 并发扩展了多少倍，每个redis node 承担16384/N个槽位
Redis cluster预先分配16384个(slot)槽位，当需要在redis集群中写入一个key -value的时候，会使用CRC16(key) mod 16384之后的值，决定将key写入值哪一个槽位从而决定写入哪一个Redis节点上，从而有效解决单机瓶颈。

redis cluster架构

7.2部署redis cluster

在所有redis主机中部署

原有的172.25.250.10 172.25.250.20 172.25.250.30 作为主master

然后我们新加 172.25.250.110 172.25.250.120 172.25.250.130 作为 slave

删除172.25.250.10 20 30 上源码安装的redis

#####删掉之前源码编译的redis
[root@redis-node1 redis]# cd /root/redis-7.4.0/
[root@redis-node1 redis-7.4.0]# ls
00-RELEASENOTES     INSTALL      redis.conf              runtest-sentinel  TLS.md
BUGS                LICENSE.txt  REDISCONTRIBUTIONS.txt  SECURITY.md       utils
CODE_OF_CONDUCT.md  Makefile     runtest                 sentinel.conf
CONTRIBUTING.md     MANIFESTO    runtest-cluster         src
deps                README.md    runtest-moduleapi       tests



[root@redis-node1 redis-7.4.0]# ps aux | grep redis
root       34042  0.2  0.6 354428 11884 ?        Ssl  14:52   0:00 /usr/local/bin/redis-server *:6379
root       34149  0.0  0.1 221664  2176 pts/0    S+   14:57   0:00 grep --color=auto redis
[root@redis-node1 redis-7.4.0]# killall -9 redis-server 
[root@redis-node1 redis-7.4.0]# ps aux | grep redis
root       34155  0.0  0.1 221664  2176 pts/0    S+   14:57   0:00 grep --color=auto redis

[root@redis-node1 redis-7.4.0]# /etc/init.d/redis_6379 stop

[root@redis-node1 redis-7.4.0]# ps aux | grep redis
root       34259  0.0  0.1 221664  2176 pts/0    S+   14:58   0:00 grep --color=auto redis
[root@redis-node1 redis-7.4.0]# ps aux | grep redis
root       34262  0.0  0.1 221664  2176 pts/0    S+   14:59   0:00 grep --color=auto redis

[root@redis-node1 redis-7.4.0]# make uninstall 

cd src && make uninstall
make[1]: Entering directory '/root/redis-7.4.0/src'
rm -f /usr/local/bin/{redis-server,redis-benchmark,redis-cli,redis-check-rdb,redis-check-aof,redis-sentinel}
make[1]: Leaving directory '/root/redis-7.4.0/src'

新加了172.25.250.110 120 130 三台主机为slave

原来的172.25.250.10 20 30 三台主机为master

一共6台主机

三主三从

我们在所有主机上用rpm包安装redis

[root@redis-node1 redis-7.4.0]# yum install redis -y

###编辑redis配置文件
[root@redis-node1 redis-7.4.0]# vim /etc/redis/redis.conf 
masterauth "123456" 		#集群主从认证
requirepass "123456" 		#redis登陆密码 redis-cli 命令连接redis后要用"auth 密码"进行认证
cluster-enabled yes 		#开启cluster集群功能
cluster-config-file nodes-6379.conf 	#指定集群配置文件
cluster-node-timeout 15000 				#节点加入集群的超时时间单位是ms


￥#####启动redis
[root@redis-node1 redis-7.4.0]# systemctl enable --now redis

[root@redis-node1 redis-7.4.0]# vim /etc/redis/redis.conf 
[root@redis-node1 redis-7.4.0]# systemctl restart redis
[root@redis-node1 redis-7.4.0]# netstat -antlupe | grep redis
tcp        0      0 0.0.0.0:16379           0.0.0.0:*               LISTEN      980        44
tcp        0      0 0.0.0.0:6379            0.0.0.0:*               LISTEN      980        44
tcp6       0      0 :::16379                :::*                    LISTEN      980        44
tcp6       0      0 :::6379                 :::*                    LISTEN      980        44
[root@redis-node1 redis-7.4.0]# bash


[root@redis-node1 redis-7.4.0]# redis-cli 
127.0.0.1:6379> keys *
(error) NOAUTH Authentication required.
127.0.0.1:6379> auth 123456
OK
127.0.0.1:6379> keys *
(empty array)
127.0.0.1:6379> set name ren
(error) CLUSTERDOWN Hash slot not served
127.0.0.1:6379> quit

###在10主机上拷贝redis中的/etc/redis/redis.conf到其他所有主机
[root@redis-node1 redis-7.4.0]# for i in 20 30 110 120 130; do scp/etc/redis/redis.conf root@172.25.250.$i:/etc/redis/redis.conf; done


[root@redis-node1 ~]# systemctl restart redis
[root@redis-node1 ~]# netstat -antlup | grep redis
tcp        0      0 0.0.0.0:16379           0.0.0.0:*               LISTEN      9216/redis-server * 
tcp        0      0 0.0.0.0:6379            0.0.0.0:*               LISTEN      9216/redis-server * 
tcp6       0      0 :::16379                :::*                    LISTEN      9216/redis-server * 
tcp6       0      0 :::6379                 :::*                    LISTEN      9216/redis-server * 
[root@redis-node1 ~]# netstat -antlup | grep 6379
tcp        0      0 0.0.0.0:16379           0.0.0.0:*               LISTEN      9216/redis-server * 
tcp        0      0 0.0.0.0:6379            0.0.0.0:*               LISTEN      9216/redis-server * 
tcp6       0      0 :::16379                :::*                    LISTEN      9216/redis-server * 
tcp6       0      0 :::6379                 :::*                    LISTEN      9216/redis-server * 


[root@redis-node1 ~]# redis-cli --cluster create -a 123456 \
> 172.25.250.10:6379 172.25.250.20:6379 172.25.250.30:6379 \
> 172.25.250.110:6379 172.25.250.120:6379 172.25.250.130:6379 \
> --cluster-replicas 1
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 172.25.250.120:6379 to 172.25.250.10:6379
Adding replica 172.25.250.130:6379 to 172.25.250.20:6379
Adding replica 172.25.250.110:6379 to 172.25.250.30:6379
M: 0f336dd07f8d0c58e4275d90f1f21bc698636b6b 172.25.250.10:6379
   slots:[0-5460] (5461 slots) master
M: 4e79ba7bf9c1eb8c0b1b31f98200611cb677f412 172.25.250.20:6379
   slots:[5461-10922] (5462 slots) master
M: 843ce27571c8aeeb2855a8a219607ba34df1938a 172.25.250.30:6379
   slots:[10923-16383] (5461 slots) master
S: 32d0f40b0643f54edce632825dc5cd1e2c4d721c 172.25.250.110:6379
   replicates 843ce27571c8aeeb2855a8a219607ba34df1938a
S: bad0965394102cfceee927593ba470b6f5382162 172.25.250.120:6379
   replicates 0f336dd07f8d0c58e4275d90f1f21bc698636b6b
S: 0bc1d271194bbaf7d182a89d4e819281aea79165 172.25.250.130:6379
   replicates 4e79ba7bf9c1eb8c0b1b31f98200611cb677f412
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join

>>> Performing Cluster Check (using node 172.25.250.10:6379)
M: 0f336dd07f8d0c58e4275d90f1f21bc698636b6b 172.25.250.10:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: 843ce27571c8aeeb2855a8a219607ba34df1938a 172.25.250.30:6379
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 32d0f40b0643f54edce632825dc5cd1e2c4d721c 172.25.250.110:6379
   slots: (0 slots) slave
   replicates 843ce27571c8aeeb2855a8a219607ba34df1938a
S: 0bc1d271194bbaf7d182a89d4e819281aea79165 172.25.250.130:6379
   slots: (0 slots) slave
   replicates 4e79ba7bf9c1eb8c0b1b31f98200611cb677f412
M: 4e79ba7bf9c1eb8c0b1b31f98200611cb677f412 172.25.250.20:6379
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: bad0965394102cfceee927593ba470b6f5382162 172.25.250.120:6379
   slots: (0 slots) slave
   replicates 0f336dd07f8d0c58e4275d90f1f21bc698636b6b
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

####查看状态
[root@redis-node1 ~]# redis-cli -a 123456 --cluster check 172.25.250.10:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.25.250.10:6379 (0f336dd0...) -> 0 keys | 5461 slots | 1 slaves.
172.25.250.30:6379 (843ce275...) -> 0 keys | 5461 slots | 1 slaves.
172.25.250.20:6379 (4e79ba7b...) -> 0 keys | 5462 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 172.25.250.10:6379)
M: 0f336dd07f8d0c58e4275d90f1f21bc698636b6b 172.25.250.10:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: 843ce27571c8aeeb2855a8a219607ba34df1938a 172.25.250.30:6379
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 32d0f40b0643f54edce632825dc5cd1e2c4d721c 172.25.250.110:6379
   slots: (0 slots) slave
   replicates 843ce27571c8aeeb2855a8a219607ba34df1938a
S: 0bc1d271194bbaf7d182a89d4e819281aea79165 172.25.250.130:6379
   slots: (0 slots) slave
   replicates 4e79ba7bf9c1eb8c0b1b31f98200611cb677f412
M: 4e79ba7bf9c1eb8c0b1b31f98200611cb677f412 172.25.250.20:6379
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: bad0965394102cfceee927593ba470b6f5382162 172.25.250.120:6379
   slots: (0 slots) slave
   replicates 0f336dd07f8d0c58e4275d90f1f21bc698636b6b
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.


[root@redis-node1 ~]# redis-cli -a 123456 --cluster info 172.25.250.10:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.25.250.10:6379 (0f336dd0...) -> 0 keys | 5461 slots | 1 slaves.
172.25.250.30:6379 (843ce275...) -> 0 keys | 5461 slots | 1 slaves.
172.25.250.20:6379 (4e79ba7b...) -> 0 keys | 5462 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.

写入数据

[root@redis-node1 ~]# redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> set name lee
(error) MOVED 5798 172.25.250.20:6379       ###指向172.25.250.20
127.0.0.1:6379> get name
(error) MOVED 5798 172.25.250.20:6379
127.0.0.1:6379> set name ren
(error) MOVED 5798 172.25.250.20:6379
127.0.0.1:6379> quit


####我们去172.25.250.20看
[root@redis-node2 ~]# redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> get name
(nil)
127.0.0.1:6379> set name lee
OK
127.0.0.1:6379> get name
"lee"

7.3集群扩容

添加主机

redis-node4 ：172.25.250.40

redis-node140 ：172.25.250.140

下载redis----两台都下载

yum install redis -y

####添加master
[root@redis-node1 ~]# redis-cli -a 123456 --cluster add-node 
172.25.250.40:6379 172.25.250.10:6379

###master添加后没有槽位
###分配槽位
[root@redis-node1 ~]#  redis-cli -a 123456 --cluster reshard 
172.25.250.10:6379     ###分配4096个槽位    all：所有主机都分配给他

####查看可以看到172.25.250.40有槽位了
[root@redis-node1 ~]# redis-cli -a 123456 --cluster check 172.25.250.10:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.25.250.10:6379 (0f336dd0...) -> 0 keys | 4096 slots | 1 slaves.
172.25.250.40:6379 (1bffc87a...) -> 1 keys | 4096 slots | 0 slaves.
172.25.250.30:6379 (843ce275...) -> 0 keys | 4096 slots | 1 slaves.
172.25.250.20:6379 (4e79ba7b...) -> 0 keys | 4096 slots | 1 slaves.
[OK] 1 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 172.25.250.10:6379)
M: 0f336dd07f8d0c58e4275d90f1f21bc698636b6b 172.25.250.10:6379
   slots:[1365-5460] (4096 slots) master
   1 additional replica(s)
M: 1bffc87a10c9c714ea3c2274bccec7b06d584444 172.25.250.40:6379
   slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
M: 843ce27571c8aeeb2855a8a219607ba34df1938a 172.25.250.30:6379
   slots:[12288-16383] (4096 slots) master
   1 additional replica(s)
S: 32d0f40b0643f54edce632825dc5cd1e2c4d721c 172.25.250.110:6379
   slots: (0 slots) slave
   replicates 843ce27571c8aeeb2855a8a219607ba34df1938a
S: 0bc1d271194bbaf7d182a89d4e819281aea79165 172.25.250.130:6379
   slots: (0 slots) slave
   replicates 4e79ba7bf9c1eb8c0b1b31f98200611cb677f412
M: 4e79ba7bf9c1eb8c0b1b31f98200611cb677f412 172.25.250.20:6379
   slots:[6827-10922] (4096 slots) master
   1 additional replica(s)
S: bad0965394102cfceee927593ba470b6f5382162 172.25.250.120:6379
   slots: (0 slots) slave
   replicates 0f336dd07f8d0c58e4275d90f1f21bc698636b6b
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

###添加172.25.250.40的slave，所以cluster-master-id 必须是要172.25.250.40的id
[root@redis-node1 ~]# redis-cli -a 123456 --cluster add-node 172.25.250.140:6379 172.25.250.10:6379 --cluster-slave --cluster-master-id 1bffc87a10c9c714ea3c2274bccec7b06d584444

Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Adding node 172.25.250.140:6379 to cluster 172.25.250.10:6379
>>> Performing Cluster Check (using node 172.25.250.10:6379)
M: 0f336dd07f8d0c58e4275d90f1f21bc698636b6b 172.25.250.10:6379
   slots:[1365-5460] (4096 slots) master
   1 additional replica(s)
M: 1bffc87a10c9c714ea3c2274bccec7b06d584444 172.25.250.40:6379
   slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
M: 843ce27571c8aeeb2855a8a219607ba34df1938a 172.25.250.30:6379
   slots:[12288-16383] (4096 slots) master
   1 additional replica(s)
S: 32d0f40b0643f54edce632825dc5cd1e2c4d721c 172.25.250.110:6379
   slots: (0 slots) slave
   replicates 843ce27571c8aeeb2855a8a219607ba34df1938a
S: 0bc1d271194bbaf7d182a89d4e819281aea79165 172.25.250.130:6379
   slots: (0 slots) slave
   replicates 4e79ba7bf9c1eb8c0b1b31f98200611cb677f412
M: 4e79ba7bf9c1eb8c0b1b31f98200611cb677f412 172.25.250.20:6379
   slots:[6827-10922] (4096 slots) master
   1 additional replica(s)
S: bad0965394102cfceee927593ba470b6f5382162 172.25.250.120:6379
   slots: (0 slots) slave
   replicates 0f336dd07f8d0c58e4275d90f1f21bc698636b6b
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 172.25.250.140:6379 to make it join the cluster.
Waiting for the cluster to join

>>> Configure node as replica of 172.25.250.40:6379.
[OK] New node added correctly.


####查看
[root@redis-node1 ~]# redis-cli -a 123456 --cluster check 172.25.250.10:6379


Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.25.250.10:6379 (0f336dd0...) -> 0 keys | 4096 slots | 1 slaves.
172.25.250.40:6379 (1bffc87a...) -> 1 keys | 4096 slots | 1 slaves.
172.25.250.30:6379 (843ce275...) -> 0 keys | 4096 slots | 1 slaves.
172.25.250.20:6379 (4e79ba7b...) -> 0 keys | 4096 slots | 1 slaves.
[OK] 1 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 172.25.250.10:6379)
M: 0f336dd07f8d0c58e4275d90f1f21bc698636b6b 172.25.250.10:6379
   slots:[1365-5460] (4096 slots) master
   1 additional replica(s)
M: 1bffc87a10c9c714ea3c2274bccec7b06d584444 172.25.250.40:6379
   slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
   1 additional replica(s)
M: 843ce27571c8aeeb2855a8a219607ba34df1938a 172.25.250.30:6379
   slots:[12288-16383] (4096 slots) master
   1 additional replica(s)
S: 32d0f40b0643f54edce632825dc5cd1e2c4d721c 172.25.250.110:6379
   slots: (0 slots) slave
   replicates 843ce27571c8aeeb2855a8a219607ba34df1938a
S: 0bc1d271194bbaf7d182a89d4e819281aea79165 172.25.250.130:6379
   slots: (0 slots) slave
   replicates 4e79ba7bf9c1eb8c0b1b31f98200611cb677f412
M: 4e79ba7bf9c1eb8c0b1b31f98200611cb677f412 172.25.250.20:6379
   slots:[6827-10922] (4096 slots) master
   1 additional replica(s)
S: 43b3521bc9bd18aba373231680da23013bc8fffe 172.25.250.140:6379
   slots: (0 slots) slave
   replicates 1bffc87a10c9c714ea3c2274bccec7b06d584444
S: bad0965394102cfceee927593ba470b6f5382162 172.25.250.120:6379
   slots: (0 slots) slave
   replicates 0f336dd07f8d0c58e4275d90f1f21bc698636b6b
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

####可以看到4主4从OK

7.4clsuter集群维护

添加节点的时候是先添加node节点到集群，然后分配槽位，删除节点的操作与添加节点的操作正好相反，是先将被删除的Redis node上的槽位迁移到集群中的其他Redis node节点上，然后再将其删除，如果一个Redis node节点上的槽位没有被完全迁移，删除该node的时候会提示有数据且无法删除。

如果想要删除掉172.25.250.40

[root@redis-node1 ~]# redis-cli -a 123456 --cluster del-node 172.25.250.140:6379 43b3521bc9bd18aba373231680da23013bc8fffe

Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Removing node 43b3521bc9bd18aba373231680da23013bc8fffe from cluster 172.25.250.140:6379
>>> Sending CLUSTER FORGET messages to the cluster...
>>> Sending CLUSTER RESET SOFT to the deleted node.
[root@redis-node1 ~]# redis-cli -a 123456 --cluster check 172.25.250.10:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.25.250.10:6379 (0f336dd0...) -> 0 keys | 4096 slots | 1 slaves.
172.25.250.40:6379 (1bffc87a...) -> 1 keys | 4096 slots | 0 slaves.
172.25.250.30:6379 (843ce275...) -> 0 keys | 4096 slots | 1 slaves.
172.25.250.20:6379 (4e79ba7b...) -> 0 keys | 4096 slots | 1 slaves.
[OK] 1 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 172.25.250.10:6379)
M: 0f336dd07f8d0c58e4275d90f1f21bc698636b6b 172.25.250.10:6379
   slots:[1365-5460] (4096 slots) master
   1 additional replica(s)
M: 1bffc87a10c9c714ea3c2274bccec7b06d584444 172.25.250.40:6379
   slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
M: 843ce27571c8aeeb2855a8a219607ba34df1938a 172.25.250.30:6379
   slots:[12288-16383] (4096 slots) master
   1 additional replica(s)
S: 32d0f40b0643f54edce632825dc5cd1e2c4d721c 172.25.250.110:6379
   slots: (0 slots) slave
   replicates 843ce27571c8aeeb2855a8a219607ba34df1938a
S: 0bc1d271194bbaf7d182a89d4e819281aea79165 172.25.250.130:6379
   slots: (0 slots) slave
   replicates 4e79ba7bf9c1eb8c0b1b31f98200611cb677f412
M: 4e79ba7bf9c1eb8c0b1b31f98200611cb677f412 172.25.250.20:6379
   slots:[6827-10922] (4096 slots) master
   1 additional replica(s)
S: bad0965394102cfceee927593ba470b6f5382162 172.25.250.120:6379
   slots: (0 slots) slave
   replicates 0f336dd07f8d0c58e4275d90f1f21bc698636b6b
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.


####清除槽位
###把172.25.250.40的槽位往172.25.250.10上迁移
[root@redis-node1 ~]# redis-cli -a 123456 --cluster reshard 172.25.250.10:6379

Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing Cluster Check (using node 172.25.250.10:6379)
M: 0f336dd07f8d0c58e4275d90f1f21bc698636b6b 172.25.250.10:6379
   slots:[1365-5460] (4096 slots) master
   1 additional replica(s)
M: 1bffc87a10c9c714ea3c2274bccec7b06d584444 172.25.250.40:6379
   slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
M: 843ce27571c8aeeb2855a8a219607ba34df1938a 172.25.250.30:6379
   slots:[12288-16383] (4096 slots) master
   1 additional replica(s)
S: 32d0f40b0643f54edce632825dc5cd1e2c4d721c 172.25.250.110:6379
   slots: (0 slots) slave
   replicates 843ce27571c8aeeb2855a8a219607ba34df1938a
S: 0bc1d271194bbaf7d182a89d4e819281aea79165 172.25.250.130:6379
   slots: (0 slots) slave
   replicates 4e79ba7bf9c1eb8c0b1b31f98200611cb677f412
M: 4e79ba7bf9c1eb8c0b1b31f98200611cb677f412 172.25.250.20:6379
   slots:[6827-10922] (4096 slots) master
   1 additional replica(s)
S: bad0965394102cfceee927593ba470b6f5382162 172.25.250.120:6379
   slots: (0 slots) slave
   replicates 0f336dd07f8d0c58e4275d90f1f21bc698636b6b
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 4096
What is the receiving node ID? 0f336dd07f8d0c58e4275d90f1f21bc698636b6b
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
  
####这里是172.25.250.40的id
Source node #1: 1bffc87a10c9c714ea3c2274bccec7b06d584444
Source node #2: done

###检测
####40上的槽位没有了
[root@redis-node1 ~]# redis-cli -a 123456 --cluster check 172.25.250.10:6379

Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.25.250.10:6379 (0f336dd0...) -> 1 keys | 8192 slots | 2 slaves.
172.25.250.30:6379 (843ce275...) -> 0 keys | 4096 slots | 1 slaves.
172.25.250.20:6379 (4e79ba7b...) -> 0 keys | 4096 slots | 1 slaves.
[OK] 1 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 172.25.250.10:6379)
M: 0f336dd07f8d0c58e4275d90f1f21bc698636b6b 172.25.250.10:6379
   slots:[0-6826],[10923-12287] (8192 slots) master
   2 additional replica(s)
S: 1bffc87a10c9c714ea3c2274bccec7b06d584444 172.25.250.40:6379
   slots: (0 slots) slave
   replicates 0f336dd07f8d0c58e4275d90f1f21bc698636b6b
M: 843ce27571c8aeeb2855a8a219607ba34df1938a 172.25.250.30:6379
   slots:[12288-16383] (4096 slots) master
   1 additional replica(s)
S: 32d0f40b0643f54edce632825dc5cd1e2c4d721c 172.25.250.110:6379
   slots: (0 slots) slave
   replicates 843ce27571c8aeeb2855a8a219607ba34df1938a
S: 0bc1d271194bbaf7d182a89d4e819281aea79165 172.25.250.130:6379
   slots: (0 slots) slave
   replicates 4e79ba7bf9c1eb8c0b1b31f98200611cb677f412
M: 4e79ba7bf9c1eb8c0b1b31f98200611cb677f412 172.25.250.20:6379
   slots:[6827-10922] (4096 slots) master
   1 additional replica(s)
S: bad0965394102cfceee927593ba470b6f5382162 172.25.250.120:6379
   slots: (0 slots) slave
   replicates 0f336dd07f8d0c58e4275d90f1f21bc698636b6b
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

####删除40，也是master
[root@redis-node1 ~]# redis-cli -a 123456 --cluster del-node 172.25.250.40:6379 1bffc87a10c9c714ea3c2274bccec7b06d584444

Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Removing node 1bffc87a10c9c714ea3c2274bccec7b06d584444 from cluster 172.25.250.40:6379
>>> Sending CLUSTER FORGET messages to the cluster...
>>> Sending CLUSTER RESET SOFT to the deleted node.


######检测  40没了
[root@redis-node1 ~]# redis-cli -a 123456 --cluster check 172.25.250.10:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.25.250.10:6379 (0f336dd0...) -> 1 keys | 8192 slots | 1 slaves.
172.25.250.30:6379 (843ce275...) -> 0 keys | 4096 slots | 1 slaves.
172.25.250.20:6379 (4e79ba7b...) -> 0 keys | 4096 slots | 1 slaves.
[OK] 1 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 172.25.250.10:6379)
M: 0f336dd07f8d0c58e4275d90f1f21bc698636b6b 172.25.250.10:6379
   slots:[0-6826],[10923-12287] (8192 slots) master
   1 additional replica(s)
M: 843ce27571c8aeeb2855a8a219607ba34df1938a 172.25.250.30:6379
   slots:[12288-16383] (4096 slots) master
   1 additional replica(s)
S: 32d0f40b0643f54edce632825dc5cd1e2c4d721c 172.25.250.110:6379
   slots: (0 slots) slave
   replicates 843ce27571c8aeeb2855a8a219607ba34df1938a
S: 0bc1d271194bbaf7d182a89d4e819281aea79165 172.25.250.130:6379
   slots: (0 slots) slave
   replicates 4e79ba7bf9c1eb8c0b1b31f98200611cb677f412
M: 4e79ba7bf9c1eb8c0b1b31f98200611cb677f412 172.25.250.20:6379
   slots:[6827-10922] (4096 slots) master
   1 additional replica(s)
S: bad0965394102cfceee927593ba470b6f5382162 172.25.250.120:6379
   slots: (0 slots) slave
   replicates 0f336dd07f8d0c58e4275d90f1f21bc698636b6b
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.