安装部署
添加bitnami仓库并查找redis
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update
[kmning@k8s-register-node ~]$ helm search repo redis
NAME CHART VERSION APP VERSION DESCRIPTION
bitnami/redis 17.10.1 7.0.11 Redis(R) is an open source, advanced key-value ...
bitnami/redis-cluster 8.4.4 7.0.11 Redis(R) is an open source, scalable, distribut...
拉取chat到本地helm pull bitnami/redis-cluster --version 8.4.4
tar -zxvf redis-cluster-8.4.4.tgz
cp redis-cluster/values.yaml ./values-test.yaml
对本地values-test.yaml进行修改，配置非常多，根据实际情况进行修改，比如我主要修改了如下内容
global:
imageRegistry: "xxx.com:443"
## E.g.
## imagePullSecrets:
## - myRegistryKeySecretName
##
imagePullSecrets: []
storageClass: "managed-nfs-storage"
redis:
password: "pwd"
所有用到image的地方改成私服
image:
registry: xxx:443
repository: lib-proxy/bitnami/redis-cluster
## Bitnami Redis® image tag
## ref: https://github.com/bitnami/containers/tree/main/bitnami/redis#supported-tags-and-respective-dockerfile-links
##
tag: 7.0.11-debian-11-r0
persistence:
## @param persistence.enabled Enable persistence on Redis®
## If enabled, nodes are using Persistent Volume Claims
## If disabled, an emptyDir volume is used
##
enabled: true
## @param persistence.path Path to mount the volume at, to use other images Redis® images.
##
path: /bitnami/redis/data
## @param persistence.subPath The subdirectory of the volume to mount to, useful in dev environments and one PV for multiple services
##
subPath: ""
## @param persistence.storageClass Storage class of backing PVC
## If defined, storageClassName: <storageClass>
## If set to "-", storageClassName: "", which disables dynamic provisioning
## If undefined (the default) or set to null, no storageClassName spec is
## set, choosing the default provisioner. (gp2 on AWS, standard on
## GKE, AWS & OpenStack)
##
storageClass: "managed-nfs-storage"
helm安装redis集群
kubectl create ns redis
helm -n redis install redis-cluster redis-cluster-8.4.4.tgz -f values-test.yaml \
--set useBundledSystemChart=true
安装后打印
kmning@k8s-master-1:~/redis-k8s-cluster$ helm -n redis install redis-cluster redis-cluster-8.4.4.tgz -f values-test.yaml \
> --set useBundledSystemChart=true
NAME: redis-cluster
LAST DEPLOYED: Thu Apr 27 08:44:02 2023
NAMESPACE: redis
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: redis-cluster
CHART VERSION: 8.4.4
APP VERSION: 7.0.11** Please be patient while the chart is being deployed **
To get your password run:
export REDIS_PASSWORD=$(kubectl get secret --namespace "redis" redis-cluster -o jsonpath="{.data.redis-password}" | base64 -d)
You have deployed a Redis® Cluster accessible only from within you Kubernetes Cluster.INFO: The Job to create the cluster will be created.To connect to your Redis® cluster:
1. Run a Redis® pod that you can use as a client:
kubectl run --namespace redis redis-cluster-client --rm --tty -i --restart='Never' \
--env REDIS_PASSWORD=$REDIS_PASSWORD \
--image k8s-register-node.com:443/lib-proxy/bitnami/redis-cluster:7.0.11-debian-11-r0 -- bash
2. Connect using the Redis® CLI:
redis-cli -c -h redis-cluster -a $REDIS_PASSWORD
查看已安装chat
kmning@k8s-master-1:~/redis-k8s-cluster$ helm -n redis list
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
redis-cluster redis 1 2023-04-27 08:33:55.88999883 +0000 UTC deployed redis-cluster-8.4.4 7.0.11
查看服务部署情况
kmning@k8s-master-1:~/redis-k8s-cluster$ kubectl get sts -n redis
NAME READY AGE
redis-cluster 6/6 4m27s
euht@k8s-master-1:~/redis-k8s-cluster$ kubectl get pods -n redis
NAME READY STATUS RESTARTS AGE
redis-cluster-0 1/1 Running 1 (3m26s ago) 4m33s
redis-cluster-1 1/1 Running 1 (3m18s ago) 4m33s
redis-cluster-2 1/1 Running 0 4m33s
redis-cluster-3 1/1 Running 0 4m33s
redis-cluster-4 1/1 Running 1 (3m25s ago) 4m33s
redis-cluster-5 1/1 Running 1 (3m6s ago) 4m33s
euht@k8s-master-1:~/redis-k8s-cluster$ kubectl get svc -n redis
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
redis-cluster ClusterIP 10.43.252.164 <none> 6379/TCP 5m23s
redis-cluster-headless ClusterIP None <none> 6379/TCP,16379/TCP 5m24s
kmning@k8s-master-1:~/redis-k8s-cluster$ kubectl get pvc -n redis
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
redis-data-redis-cluster-0 Bound pvc-42ac9dea-3dda-4c43-90c6-4cd59f5af8b9 8Gi RWO managed-nfs-storage 15m
redis-data-redis-cluster-1 Bound pvc-3c5cc893-dc6d-4f2a-b485-00544564a5b5 8Gi RWO managed-nfs-storage 15m
redis-data-redis-cluster-2 Bound pvc-064448ee-240e-4658-b161-c96da01c977d 8Gi RWO managed-nfs-storage 15m
redis-data-redis-cluster-3 Bound pvc-198311a6-2c25-41d6-ae0f-6e6c055b1a20 8Gi RWO managed-nfs-storage 15m
redis-data-redis-cluster-4 Bound pvc-419b04c2-ff80-4907-aafc-3adaf33242a2 8Gi RWO managed-nfs-storage 15m
redis-data-redis-cluster-5 Bound pvc-92a0de8f-d073-4430-9c64-a06e86aa3041 8Gi RWO managed-nfs-storage 15m
看起来一切正常。默认集群主节点为redis-cluster-0、redis-cluster-2、redis-cluster-4，而集群从节点为redis-cluster-1、redis-cluster-3、redis-cluster-5。
服务验证
kubectl exec -it redis-cluster-0 bash -n redis
I have no name!@redis-cluster-client:/$ redis-cli -c -h redis-cluster -a $REDIS_PASSWORD
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
redis-cluster:6379> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:3
cluster_stats_messages_ping_sent:1022
cluster_stats_messages_pong_sent:1006
cluster_stats_messages_meet_sent:1
cluster_stats_messages_sent:2029
cluster_stats_messages_ping_received:1006
cluster_stats_messages_pong_received:1023
cluster_stats_messages_received:2029
total_cluster_links_buffer_limit_exceeded:0
redis-cluster:6379> cluster nodes
dcc21275c8dc404d2fd3d6c18c719face4640e07 10.42.2.38:6379@16379 slave b118e1f8eb353849abe08749540235264887918c 0 1682586093120 3 connected
358ddf185fe7b9f3c88745a15edeb7f513c6e695 10.42.4.15:6379@16379 slave 10eb0915585400a27b725f721d67b7bb11e8babf 0 1682586094125 2 connected
922c7d7ce319276b4693192372d8b217b608d733 10.42.3.17:6379@16379 slave 012396047e02fc8c890dfb838dbb6e3d8e497eb9 0 1682586088098 1 connected
012396047e02fc8c890dfb838dbb6e3d8e497eb9 10.42.0.38:6379@16379 master - 0 1682586092117 1 connected 0-5460
10eb0915585400a27b725f721d67b7bb11e8babf 10.42.1.30:6379@16379 master - 0 1682586095129 2 connected 5461-10922
b118e1f8eb353849abe08749540235264887918c 10.42.5.13:6379@16379 myself,master - 0 1682586045000 3 connected 10923-16383
redis-cluster:6379>
从日志可以看出，三主三从的分片集群已完成搭建。
客户端使用
通过上面的安装，我们已经将Service暴露出来
kmning@k8s-master-1:~/redis-k8s-cluster$ kubectl get svc -n redis
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
redis-cluster ClusterIP 10.43.252.164 <none> 6379/TCP 5m23s
redis-cluster-headless ClusterIP None <none> 6379/TCP,16379/TCP 5m24s
可以看出，暴露了一个可负载均衡访问的Service和一个无头Service，可根据需要去使用。正常来讲，我们只需要以分片集群的方式连接可负载均衡的Service即可。
如本文的情况，redis cluster的每个节点都是一个跑在k8s里面的pod，这些pod并不能被外部直接访问，而是通过ingress等方法对外暴露一个访问接口，即只有一个统一的ip：port给外部访问。经由k8s的调度，对这个统一接口的访问会被发送到redis集群的某个节点。这时候对redis的用户来说，看起来这就像是一个单节点的redis。但是，此时无论是直接使用命令行工具redis-cli，还是某种语言的sdk，还是需要按照集群来配置redis的连接信息，才能正确连接，例如
spring:
redis:
cluster:
nodes: redis-cluster.redis.svc.cluster.local:6379
password: yourpwd
当然，这里也可以使用无头Service去灵活连接，把nodes的内容改下如下即可
redis-cluster-0.redis-cluster-headless.redis.svc.cluster.local,redis-cluster-1.redis-cluster-headless.redis.svc.cluster.local,redis-cluster-2.redis-cluster-headless.redis.svc.cluster.local,redis-cluster-3.redis-cluster-headless.redis.svc.cluster.local,redis-cluster-4.redis-cluster-headless.redis.svc.cluster.local,redis-cluster-5.redis-cluster-headless.redis.svc.cluster.local