Prometheus之监控K8S集群(部署在k8s集群内部)

复制代码
	1. prometheus-operator概述
prometheus-operator可以一键实现对K8S集群的监控。

GitHub地址:
	https://github.com/prometheus-operator/kube-prometheus


	2.在K8S集群部署prometheus
		2.1 基于K8S版本选择合适的prometheus-operator
参考地址:
	https://github.com/prometheus-operator/kube-prometheus#compatibility

		2.2 下载软件包
wget https://github.com/prometheus-operator/kube-prometheus/archive/refs/tags/v0.11.0.tar.gz
	
		2.3 解压软件包
[root@master231 ~]# tar xf kube-prometheus-0.11.0.tar.gz -C /zqylinux/softwares/


		2.4 进入到prometheus-operator主目录
[root@master231 ~]# cd /zqylinux/softwares/kube-prometheus-0.11.0/
[root@master231 kube-prometheus-0.11.0]# 
[root@master231 kube-prometheus-0.11.0]# ll
total 220
drwxrwxr-x 11 root root  4096 Jun 17  2022 ./
drwxr-xr-x  4 root root  4096 Aug 20 10:48 ../
-rwxrwxr-x  1 root root   679 Jun 17  2022 build.sh*
-rw-rw-r--  1 root root 11421 Jun 17  2022 CHANGELOG.md
-rw-rw-r--  1 root root  2020 Jun 17  2022 code-of-conduct.md
-rw-rw-r--  1 root root  3782 Jun 17  2022 CONTRIBUTING.md
drwxrwxr-x  5 root root  4096 Jun 17  2022 developer-workspace/
drwxrwxr-x  4 root root  4096 Jun 17  2022 docs/
-rw-rw-r--  1 root root  2273 Jun 17  2022 example.jsonnet
drwxrwxr-x  7 root root  4096 Jun 17  2022 examples/
drwxrwxr-x  3 root root  4096 Jun 17  2022 experimental/
drwxrwxr-x  4 root root  4096 Jun 17  2022 .github/
-rw-rw-r--  1 root root   129 Jun 17  2022 .gitignore
-rw-rw-r--  1 root root  1474 Jun 17  2022 .gitpod.yml
-rw-rw-r--  1 root root  2302 Jun 17  2022 go.mod
-rw-rw-r--  1 root root 71172 Jun 17  2022 go.sum
drwxrwxr-x  3 root root  4096 Jun 17  2022 jsonnet/
-rw-rw-r--  1 root root   206 Jun 17  2022 jsonnetfile.json
-rw-rw-r--  1 root root  5130 Jun 17  2022 jsonnetfile.lock.json
-rw-rw-r--  1 root root  1644 Jun 17  2022 kubescape-exceptions.json
-rw-rw-r--  1 root root  4773 Jun 17  2022 kustomization.yaml
-rw-rw-r--  1 root root 11325 Jun 17  2022 LICENSE
-rw-rw-r--  1 root root  3379 Jun 17  2022 Makefile
drwxrwxr-x  3 root root  4096 Jun 17  2022 manifests/
-rw-rw-r--  1 root root   221 Jun 17  2022 .mdox.validate.yaml
-rw-rw-r--  1 root root  8253 Jun 17  2022 README.md
-rw-rw-r--  1 root root  4463 Jun 17  2022 RELEASE.md
drwxrwxr-x  2 root root  4096 Jun 17  2022 scripts/
drwxrwxr-x  3 root root  4096 Jun 17  2022 tests/
[root@master231 kube-prometheus-0.11.0]# 


	2.5 安装自定义资源
		2.5.1 修改配置文件
温馨提示:
	如果镜像下载失败,可进行如下操作:

[root@master231 kube-prometheus-0.11.0]# vim manifests/prometheusAdapter-deployment.yaml
...
        # image: k8s.gcr.io/prometheus-adapter/prometheus-adapter:v0.9.1
        image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/k8s.gcr.io/prometheus-adapter/prometheus-adapter:v0.9.1
...



[root@master231 kube-prometheus-0.11.0]# vim manifests/kubeStateMetrics-deployment.yaml
...
        # image: k8s.gcr.io/kube-state-metrics/kube-state-metrics:v2.5.0
        image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/k8s.gcr.io/kube-state-metrics/kube-state-metrics:v2.5.0
···

[root@master231 kube-prometheus-0.11.0]# vim manifests/blackboxExporter-deployment.yaml
···
        # image: jimmidyson/configmap-reload:v0.5.0
        image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/jimmidyson/configmap-reload:v0.5.0
···

[root@master231 kube-prometheus-0.11.0]# vim manifests/grafana-deployment.yaml
···
        # image: grafana/grafana:8.5.5
        image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/grafana/grafana:8.5.5
···


以上操作是修改镜像源


将grafana和prometheus-service端口使用nodeport将端口暴露:

[root@master231 kube-prometheus-0.11.0]# vim manifests/grafana-service.yaml
....
spec:
  ...
  type: NodePort
  ports:
  - name: http
    port: 3000
    targetPort: http
    nodePort: 30080
···


[root@master231 kube-prometheus-0.11.0]# vim manifests/prometheus-service.yaml
...
spec:
  type: NodePort
  ports:
  - name: web
    port: 9090
    targetPort: web
    nodePort: 30090
···

		2.5.2 部署服务(直接复制粘贴,看好路径)
[root@master231 kube-prometheus-0.11.0]# kubectl apply --server-side -f manifests/setup
[root@master231 kube-prometheus-0.11.0]# kubectl wait \
	--for condition=Established \
	--all CustomResourceDefinition \
	--namespace=monitoring
[root@master231 kube-prometheus-0.11.0]# kubectl apply -f manifests/
 

[root@master231 kube-prometheus-0.11.0]# kubectl get pods -n monitoring  -o wide
NAME                                  READY   STATUS    RESTARTS   AGE   IP            NODE        NOMINATED NODE   READINESS GATES
alertmanager-main-0                   2/2     Running   0          16m   10.100.1.12   worker232   <none>           <none>
alertmanager-main-1                   2/2     Running   0          16m   10.100.2.13   worker233   <none>           <none>
alertmanager-main-2                   2/2     Running   0          16m   10.100.1.14   worker232   <none>           <none>
blackbox-exporter-746c64fd88-8qsjv    3/3     Running   0          14m   10.100.1.15   worker232   <none>           <none>
grafana-5fc7f9f55d-xdwt9              1/1     Running   0          16m   10.100.1.9    worker232   <none>           <none>
kube-state-metrics-5c5cfbf465-8gmfb   3/3     Running   0          16m   10.100.2.10   worker233   <none>           <none>
node-exporter-dqpkf                   2/2     Running   0          16m   10.0.0.233    worker233   <none>           <none>
node-exporter-ff8vw                   2/2     Running   0          16m   10.0.0.231    master231   <none>           <none>
node-exporter-ppf7v                   2/2     Running   0          16m   10.0.0.232    worker232   <none>           <none>
prometheus-adapter-5b479bc754-dnqgt   1/1     Running   0          16m   10.100.2.11   worker233   <none>           <none>
prometheus-adapter-5b479bc754-p78d2   1/1     Running   0          16m   10.100.1.11   worker232   <none>           <none>
prometheus-k8s-0                      2/2     Running   0          16m   10.100.2.14   worker233   <none>           <none>
prometheus-k8s-1                      2/2     Running   0          16m   10.100.1.13   worker232   <none>           <none>
prometheus-operator-f59c8b954-vsqmj   2/2     Running   0          16m   10.100.2.12   worker233   <none>           <none>
[root@master231 kube-prometheus-0.11.0]# 


	2.6 修改收件人和发件人信息(可选操作)
[root@master231 kube-prometheus-0.11.0]# vim manifests/alertmanager-secret.yaml 
里面记录了alertmanager的收件人和发件人信息。

 

	2.7 访问WebUI 
http://10.0.0.231:30080/
http://10.0.0.231:30090


	2.8 查看内置的模板 
点击最上方的Default即可