部署redis
创建ConfigMap
- 首先创建命名空间
arduino
kubectl create namespace ospp
- 编写redis-config.yaml文件,内容如下:
yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: redis-config
namespace: ospp
labels:
app: redis
data:
redis.conf: |-
dir /srv
port 6379
bind 0.0.0.0
appendonly yes
daemonize no
#protected-mode no
requirepass test
pidfile /srv/redis-6379.pid
- 使用如下命令创建configmap
csharp
[root@master ospp]# kubectl apply -f redis-config.yaml
configmap/redis-config created
[root@master ospp]#
创建Redis
- 编写redis.yaml文件,内容如下:
yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis
namespace: ospp
labels:
app: redis
spec:
replicas: 1
selector:
matchLabels:
app: redis
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: redis:5.0.7
command:
- "sh"
- "-c"
- "redis-server /usr/local/redis/redis.conf"
ports:
- containerPort: 6379
resources:
limits:
cpu: 1000m
memory: 1024Mi
requests:
cpu: 1000m
memory: 1024Mi
livenessProbe:
tcpSocket:
port: 6379
initialDelaySeconds: 300
timeoutSeconds: 1
periodSeconds: 10
successThreshold: 1
failureThreshold: 3
readinessProbe:
tcpSocket:
port: 6379
initialDelaySeconds: 5
timeoutSeconds: 1
periodSeconds: 10
successThreshold: 1
failureThreshold: 3
volumeMounts:
- name: config
mountPath: /usr/local/redis/redis.conf
subPath: redis.conf
volumes:
- name: config
configMap:
name: redis-config
---
apiVersion: v1
kind: Service
metadata:
name: service-redis
namespace: ospp
spec:
ports:
- port: 6379
protocol: TCP
targetPort: 6379
nodePort: 30120
selector:
app: redis
type: NodePort
- 使用如下命令创建redis
bash
[root@master ospp]# kubectl apply -f redis.yaml
deployment.apps/redis created
service/service-redis unchanged
[root@master ospp]#
- 查看创建资源如下:
sql
**➜** **~** kubectl get service,deploy,pod -n ospp -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/service-redis NodePort 10.96.215.124 <none> 6379:30120/TCP,9121:30826/TCP 21h app=redis
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
deployment.apps/redis 1/1 1 1 106m redis,redis-exporter redis:5.0.7,oliver006/redis_exporter:v1.50.0 app=redis
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/redis-85f6df986-cxx4s 2/2 Running 0 106m 10.244.4.7 my-k8s-cluster-3-worker <none> <none>
**➜** **~**
部署成功图如下:
- Deployment(工作负载)
- Service(服务)**
部署redis_exporter
创建 redis-exporter deploy 应用
yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis-exporter
namespace: monitoring
labels:
app.kubernetes.io/component: exporter
app.kubernetes.io/name: redis-exporter
app.kubernetes.io/part-of: kube-prometheus
app.kubernetes.io/version: 1.37.0
spec:
selector:
matchLabels:
app.kubernetes.io/component: exporter
app.kubernetes.io/name: redis-exporter
app.kubernetes.io/part-of: kube-prometheus
replicas: 1
template:
metadata:
labels:
app.kubernetes.io/component: exporter
app.kubernetes.io/name: redis-exporter
app.kubernetes.io/part-of: kube-prometheus
app.kubernetes.io/version: 1.37.0
spec:
containers:
- name: redis-exporter
image: oliver006/redis_exporter:v1.37.0
args:
[
"-redis.addr",
"redis://service-redis.ospp:6379", ## 这里填写账号密码
"-redis.password",
"test",
]
resources:
requests:
cpu: 20m
memory: 20Mi
limits:
cpu: 100m
memory: 30Mi
ports:
- containerPort: 9121
name: http
volumeMounts:
- name: localtime
mountPath: /etc/localtime
volumes:
- name: localtime
hostPath:
path: /usr/share/zoneinfo/Asia/Shanghai
restartPolicy: Always
创建 redis-exporter service
yaml
---
apiVersion: v1
kind: Service
metadata:
name: redis-exporter
namespace: monitoring
#labels定义很重要,后面的ServiceMonitor资源需要使用到
labels:
app.kubernetes.io/component: exporter
app.kubernetes.io/name: redis-exporter
app.kubernetes.io/part-of: kube-prometheus
app.kubernetes.io/version: 1.37.0
spec:
selector:
app.kubernetes.io/component: exporter
app.kubernetes.io/name: redis-exporter
app.kubernetes.io/part-of: kube-prometheus
type: ClusterIP
ports:
- name: http
port: 9121
targetPort: http
创建 redis-exporter servicemonitor
yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
labels:
app.kubernetes.io/component: exporter
app.kubernetes.io/name: redis-exporter
app.kubernetes.io/part-of: kube-prometheus
app.kubernetes.io/version: 1.37.0
name: redis-exporter
namespace: monitoring
spec:
endpoints: #抓取的配置
- interval: 15s #每间隔15秒抓取一次指标数据
port: http #Service定义的port的名字
relabelings: #指标的重写
- action: replace
regex: (.*)
replacement: $1
sourceLabels:
- __meta_kubernetes_pod_node_name
targetLabel: instance
scheme: http
jobLabel: app.kubernetes.io/name
selector: #选择器,这个地方要配置对应Service的labels信息
matchLabels:
app.kubernetes.io/component: exporter
app.kubernetes.io/name: redis-exporter
app.kubernetes.io/part-of: kube-prometheus
- 查看 prometheus target 添加效果:
- kuboard
说明:
-
prometheus operator是通过ServiceMonitor发现监控目标,并对其进行监控。serviceMonitor 是对service 获取数据的一种方式。**
-
promethus-operator可以通过serviceMonitor 自动识别带有某些 label 的service ,并从这些service 获取数据。
-
serviceMonitor 也是由promethus-operator 自动发现的。
-
prometheus监控过程如下:
-
部署成功图如下
-
serviceMonitor部署
3、Prometheus告警规则配置
- prometheus rule规则配置:
yaml
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
labels:
prometheus: k8s
role: alert-rules
name: redis-exporter-rules
namespace: monitoring
spec:
groups:
- name: redis-exporter
rules:
- alert: RedisDown
expr: redis_up == 0
for: 0m
labels:
severity: critical
annotations:
summary: Redis down (instance {{ $labels.instance }})
description: "Redis instance is down\n VALUE = {{ $value }}\n LABELS = {{ $labels }}"
- alert: RedisClusterFlapping
expr: changes(redis_connected_slaves[1m]) > 1
for: 2m
labels:
severity: critical
annotations:
summary: Redis cluster flapping (instance {{ $labels.instance }})
description: "Changes have been detected in Redis replica connection. This can occur when replica nodes lose connection to the master and reconnect (a.k.a flapping).\n VALUE = {{ $value }}\n LABELS = {{ $labels }}"
- alert: RedisReplicationBroken
expr: delta(redis_connected_slaves[1m]) < 0
for: 0m
labels:
severity: critical
annotations:
summary: Redis replication broken (instance {{ $labels.instance }})
description: "Redis instance lost a slave\n VALUE = {{ $value }}\n LABELS = {{ $labels }}"
- alert: RedisDisconnectedSlaves
expr: count without (instance, job) (redis_connected_slaves) - sum without (instance, job) (redis_connected_slaves) - 1 > 1
for: 0m
labels:
severity: critical
annotations:
summary: Redis disconnected slaves (instance {{ $labels.instance }})
description: "Redis not replicating for all slaves. Consider reviewing the redis replication status.\n VALUE = {{ $value }}\n LABELS = {{ $labels }}"
- alert: RedisRejectedConnections
expr: increase(redis_rejected_connections_total[1m]) > 0
for: 0m
labels:
severity: critical
annotations:
summary: Redis rejected connections (instance {{ $labels.instance }})
description: "Some connections to Redis has been rejected\n VALUE = {{ $value }}\n LABELS = {{ $labels }}"
-
prometheusRule规则配置,可以参考模板配置,模板网址如下:awesome-prometheus-alerts.grep.to/rules#redis**
-
部署成功图如下:**
Grafana部署图
- grafana dashboard地址如下:grafana.com/grafana/das...**
单节点监控,推荐模板ID为:11835;
集群方式监控,推荐模板ID为:14615
- dashboard效果图如下**
单节点监控: