Pod里面对容器进行配额管理
-
在生产环境中,内存占用通常很大,如果里面有死循环,会导致内存和cpu过大导致影响其他pod运行资源
-
需要让资源在受控的环境下运行,一般都是加上 resources limits 的配置才能达到最佳实践
-
默认cpu是0.5个cpu, 一般定义的时候参数是最高的门槛,不会超过这个设置
-
创建一个命名空间,以便将 本练习中创建的资源与集群的其余部分资源隔离
- $
kubectl create namespace cpu-stress
- $
-
创建 cpu-limit.yaml
yamlapiVersion: v1 kind: Pod metadata: name: cpu-stress-demo namespace: cpu-stress labels: name: cpu-stress-demo spec: containers: - name: nginx-stress image: registry.cn-beijing.aliyuncs.com/qingfeng666/stress args: - -cpus - "2" # 指定使用2个CPU核心进行压力测试 resources: limits: cpu: "1" # 设置CPU上限为1个核心 requests: cpu: "0.5" # 设置CPU请求为0.5个核心(可选,根据集群策略设置) restartPolicy: Never # 设置重启策略为Never,因为这是一个一次性任务
-
$
kubectl create -f cpu-limit.yaml
confpod/cpu-stress-demo created
-
$
kubectl get po -n cpu-stress -w
confNAME READY STATUS RESTARTS AGE cpu-stress-demo 1/1 Running 0 7s
-
$
kubectl get po cpu-stress-demo -n cpu-stress -o yaml
yamlapiVersion: v1 items: - apiVersion: v1 kind: Pod metadata: creationTimestamp: "2024-04-18T22:34:13Z" labels: name: cpu-stress-demo name: cpu-stress-demo namespace: cpu-stress resourceVersion: "81301" uid: f423a147-4bac-4f82-a791-0e851d67366c spec: containers: - args: - -cpus - "2" image: registry.cn-beijing.aliyuncs.com/qingfeng666/stress imagePullPolicy: Always name: nginx-stress resources: limits: cpu: "1" requests: cpu: 500m terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts: - mountPath: /var/run/secrets/kubernetes.io/serviceaccount name: kube-api-access-mg68j readOnly: true dnsPolicy: ClusterFirst enableServiceLinks: true nodeName: node2.k8s preemptionPolicy: PreemptLowerPriority priority: 0 restartPolicy: Never schedulerName: default-scheduler securityContext: {} serviceAccount: default serviceAccountName: default terminationGracePeriodSeconds: 30 tolerations: - effect: NoExecute key: node.kubernetes.io/not-ready operator: Exists tolerationSeconds: 300 - effect: NoExecute key: node.kubernetes.io/unreachable operator: Exists tolerationSeconds: 300 volumes: - name: kube-api-access-mg68j projected: defaultMode: 420 sources: - serviceAccountToken: expirationSeconds: 3607 path: token - configMap: items: - key: ca.crt path: ca.crt name: kube-root-ca.crt - downwardAPI: items: - fieldRef: apiVersion: v1 fieldPath: metadata.namespace path: namespace status: conditions: - lastProbeTime: null lastTransitionTime: "2024-04-18T22:34:13Z" status: "True" type: Initialized - lastProbeTime: null lastTransitionTime: "2024-04-18T22:34:16Z" status: "True" type: Ready - lastProbeTime: null lastTransitionTime: "2024-04-18T22:34:16Z" status: "True" type: ContainersReady - lastProbeTime: null lastTransitionTime: "2024-04-18T22:34:13Z" status: "True" type: PodScheduled containerStatuses: - containerID: docker://12668f59353306dda08a395bf7dd36c0eae699b0ed1350ec96b8ffc3705b6a5e image: registry.cn-beijing.aliyuncs.com/qingfeng666/stress:latest imageID: docker-pullable://registry.cn-beijing.aliyuncs.com/qingfeng666/stress@sha256:155d7266cb7ed6fecd34b2e4f8a25c2b21eb77723658fb4ab2db630d41118c7d lastState: {} name: nginx-stress ready: true restartCount: 0 started: true state: running: startedAt: "2024-04-18T22:34:15Z" hostIP: 10.211.55.12 phase: Running podIP: 10.244.2.12 podIPs: - ip: 10.244.2.12 qosClass: Burstable startTime: "2024-04-18T22:34:13Z" kind: List metadata: resourceVersion: "" selfLink: ""
- 从上面可以看到,没有超过1核,限制成功了
相关原理
1 )Docker namespace 隔离
- 虚拟机的隔离是非常彻底的,但是成本也是极大的
- docker 通过namespace来隔离
- $
ps aux
查看进程 - $
unshare --fork --pid --mount--proc bash
- $
ps aux
这时候再次查看,可以看到只有2个进程了 - 执行docker容器的时候,和这个类似
- $
docker run -it busybox
- 这个busybox 是一个非常小的镜像
- $
ps aux
可以看到只有2个进程
- docker和 unshare的效果一模一样
2 )CGroups 实现资源配额
- CGroups 是 control groups 的意思
- $
cd /syc/fs/cgroup/cpu
- 进入到这个目录,新建一个目录
- $
mkdir cgrous_test
- $
cd cgrous_test
进去后,发现生成了一堆文件,自动生成的 - $
cat cpu.cfs_quota_us
发现默认 -1 - $
echo 20000 > cpu.cfs_quota_us
这里20%的cpu时间 - $
echo 2754 > tasks
- $
cat tasks
可看到 2754 - $
top
查看,就发现从 死循环 100%的占用,变成了20% - 这样就可以进行限额处理了
- $
kill -9 2795
删除之
- 这个是 docker 进程实现原理
- $
docker run -it --cpu=".5" nginx /bin/sh
设置cpu配额是 50% cpu - 进入中,查看 $
/sys/fs/cgroup/cpu
目录,$cat cpu.cfs_quota_us
- 显示 50000 这个就是 .5
- 就是这个原理
- $
- 所以,这些配额限制就是调用 cgroups 的原理