k8s-pod的资源限制

一、指定内存请求和限制
二、超过容器限制的内存
三、超过整个节点容量的内存
四、限制cpu
总结

官方文档：https://kubernetes.io/zh/docs/tasks/configure-pod-container/assign-memory-resource/

一、指定内存请求和限制

创建一个容器，将会请求 100 MiB 内存，并且内存会被限制在 200 MiB 以内

yaml 复制代码

[root@k8s-1 pod]# vim memory_request_limit.yaml
apiVersion: v1
kind: Pod
metadata:
  name: memory-demo
  namespace: mem-example
spec:
  containers:
  - name: memory-demo-ctr
    image: polinux/stress
    resources:
      requests:
        memory: "100Mi"
      limits:
        memory: "200Mi"
    command: ["stress"]
    args: ["--vm", "1", "--vm-bytes", "150M", "--vm-hang", "1"]

执行

bash 复制代码

[root@k8s-1 pod]# kubectl apply -f memory_request_limit.yaml 
Error from server (NotFound): error when creating "memory_request_limit.yaml": namespaces "mem-example" not found
[root@k8s-1 pod]# kubectl get ns
NAME                   STATUS   AGE
default                Active   2d20h
kube-node-lease        Active   2d20h
kube-public            Active   2d20h
kube-system            Active   2d20h
kubernetes-dashboard   Active   2d18h

# 创建命名空间
[root@k8s-1 pod]# kubectl create namespace mem-example
namespace/mem-example created
[root@k8s-1 pod]# kubectl get ns
NAME                   STATUS   AGE
default                Active   2d20h
kube-node-lease        Active   2d20h
kube-public            Active   2d20h
kube-system            Active   2d20h
kubernetes-dashboard   Active   2d18h
mem-example            Active   4s
[root@k8s-1 pod]# kubectl apply -f memory_request_limit.yaml 
pod/memory-demo created

# 查看是否运行
[root@k8s-1 pod]# kubectl get pod -n memory-example
No resources found in memory-example namespace.
[root@k8s-1 pod]# kubectl get pod -n mem-example
NAME          READY   STATUS              RESTARTS   AGE
memory-demo   0/1     ContainerCreating   0          40s
[root@k8s-1 pod]# kubectl get pod -n mem-example
NAME          READY   STATUS    RESTARTS   AGE
memory-demo   1/1     Running   0          117s
# 查看更多信息
[root@k8s-1 pod]# kubectl get pod -n mem-example -o wide
NAME          READY   STATUS    RESTARTS   AGE     IP             NODE    NOMINATED NODE   READINESS GATES
memory-demo   1/1     Running   0          7m40s   10.224.13.68   k8s-3   <none>           <none>
[root@k8s-3 ~]# docker pull polinux/stress

获取pod的指标数据，需要用metrics

bash 复制代码

[root@k8s-1 pod]# kubectl top pod memory-demo --namespace=mem-example
error: Metrics API not available
[root@k8s-1 pod]# docker pull registry.k8s.io/metrics-server/metrics-server:v0.6.3
Error response from daemon: Head "https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/metrics-server/metrics-server/manifests/v0.6.3": dial tcp 108.177.125.82:443: connect: connection refused

# 每个节点导入镜像
[root@k8s-1 images]# ls
metrics-server-v0.6.3.tar
[root@k8s-1 images]# docker load -i metrics-server-v0.6.3.tar 
[root@k8s-2 images]# docker load -i metrics-server-v0.6.3.tar [root@k8s-3 images]# docker load -i metrics-server-v0.6.3.tar 

[root@k8s-1 pod]# kubectl apply -f components.yaml 
serviceaccount/metrics-server created
clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created
clusterrole.rbac.authorization.k8s.io/system:metrics-server created
rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created
clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created
clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created
service/metrics-server created
deployment.apps/metrics-server created
apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created
[root@k8s-1 pod]# kubectl get pod -n kube-system
NAME                                       READY   STATUS    RESTARTS      AGE
metrics-server-784768bd4b-jtnsg            1/1     Running   0             2m20s
# 获取pod指标数据
[root@k8s-1 pod]# kubectl top pod memory-demo --namespace=mem-example
NAME          CPU(cores)   MEMORY(bytes)   
memory-demo   14m          150Mi

正在使用的内存大约为 150 MiB。这大于 Pod 请求的 100 MiB，但在 Pod 限制的 200 MiB之内

二、超过容器限制的内存

当节点拥有足够的可用内存时，容器可以使用其请求的内存。但是，容器不允许使用超过其限制的内存。如果容器分配的内存超过其限制，该容器会成为被终止的候选容器。如果容器继续消耗超出其限制的内存，则终止容器。如果终止的容器可以被重启，则 kubelet 会重新启动它，就像其他任何类型的运行时失败一样

创建一个 Pod，尝试分配超出其限制的内存。该容器的内存请求为 50 MiB，内存限制为 100 MiB

yaml 复制代码

[root@k8s-1 pod]# cat memory_request_limit.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: memory-demo
  namespace: mem-example
spec:
  containers:
  - name: memory-demo-ctr
    image: polinux/stress
    resources:
      requests:
        memory: "50Mi"
      limits:
        memory: "100Mi"
    command: ["stress"]
    args: ["--vm", "1", "--vm-bytes", "260M", "--vm-hang", "1"]
[root@k8s-1 pod]# kubectl apply -f memory_request_limit.yaml

查看pod信息

bash 复制代码

[root@k8s-1 pod]# kubectl get pod -A
NAMESPACE              NAME                                         READY   STATUS              RESTARTS      AGE
mem-example            memory-demo                                  0/1     ContainerCreating   0             10s
[root@k8s-1 pod]# kubectl top pod memory-demo --namespace=mem-example
Error from server (NotFound): podmetrics.metrics.k8s.io "mem-example/memory-demo" not found
[root@k8s-1 pod]# kubectl get pod -A
NAMESPACE              NAME                                         READY   STATUS      RESTARTS      AGE
mem-example            memory-demo                                  0/1     OOMKilled   2 (36s ago)   54s
[root@k8s-1 pod]# kubectl get pod -A
NAMESPACE              NAME                                         READY   STATUS              RESTARTS      AGE
mem-example            memory-demo                                  0/1     CrashLoopBackOff    2 (20s ago)   59s

# 查看pod相关的详细信息
[root@k8s-1 pod]# kubectl get pod memory-demo --namespace=mem-example
NAME          READY   STATUS              RESTARTS   AGE
memory-demo   0/1     ContainerCreating   0          12s
[root@k8s-1 pod]# kubectl get pod memory-demo --namespace=mem-example
NAME          READY   STATUS      RESTARTS   AGE
memory-demo   0/1     OOMKilled   0          13s
[root@k8s-1 pod]# kubectl get pod memory-demo --namespace=mem-example
NAME          READY   STATUS   RESTARTS     AGE
memory-demo   0/1     Error    1 (8s ago)   19s
[root@k8s-1 pod]# kubectl get pod memory-demo --namespace=mem-example
NAME          READY   STATUS             RESTARTS     AGE
memory-demo   0/1     CrashLoopBackOff   1 (4s ago)   22s

# 详细信息，以yaml文件输出
[root@k8s-1 pod]# kubectl get pod memory-demo --output=yaml --namespace=mem-example

# 删除
[root@k8s-1 pod]# kubectl delete -f memory_request_limit.yaml 
pod "memory-demo" deleted

容器由于内存溢出（OOM）被杀掉、重启、再杀掉、再重启

三、超过整个节点容量的内存

Pod 的调度基于请求。只有当节点拥有足够满足 Pod 内存请求的内存时，才会将 Pod 调度至节点上运行

yaml 复制代码

[root@k8s-1 pod]# cat memory_request_limit.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: memory-demo
  namespace: mem-example
spec:
  containers:
  - name: memory-demo-ctr
    image: polinux/stress
    resources:
      requests:
        memory: "1000Gi"
      limits:
        memory: "1000Gi"
    command: ["stress"]
    args: ["--vm", "1", "--vm-bytes", "260M", "--vm-hang", "1"]
[root@k8s-1 pod]# kubectl apply -f memory_request_limit.yaml 
pod/memory-demo created

Pod 处于 Pending 状态。这意味着，该 Pod 没有被调度至任何节点上运行，并且它会无限期的保持该状态

bash 复制代码

[root@k8s-1 pod]# kubectl get pod memory-demo --namespace=mem-example
NAME          READY   STATUS    RESTARTS   AGE
memory-demo   0/1     Pending   0          32s
# 由于节点内存不足，该容器无法被调度
[root@k8s-1 pod]# kubectl describe pod memory-demo --namespace=mem-example|grep -A 5 Events
Events:
  Type     Reason            Age                 From               Message
  ----     ------            ----                ----               -------
  Warning  FailedScheduling  45s (x2 over 112s)  default-scheduler  0/3 nodes are available: 1 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate, 2 Insufficient memory.

四、限制cpu

yaml 复制代码

[root@k8s-1 pod]# vim cpu_request_limit.yaml
apiVersion: v1
kind: Pod
metadata:
  name: cpu-demo-2
  namespace: cpu-example
spec:
  containers:
  - name: cpu-demo-ctr-2
    image: vish/stress
    resources:
      limits:
        cpu: "100"
      requests:
        cpu: "100"
    args:
    - -cpus
    - "2"

输出显示 Pod 状态为 Pending。也就是说，Pod 未被调度到任何节点上运行，并且 Pod 将无限期地处于 Pending 状态

bash 复制代码

[root@k8s-1 pod]# kubectl create namespace cpu-example
namespace/cpu-example created
[root@k8s-1 pod]# kubectl apply -f cpu_request_limit.yaml 
pod/cpu-demo-2 created
[root@k8s-1 pod]# kubectl get pod cpu-demo-2 --namespace=cpu-example
NAME         READY   STATUS    RESTARTS   AGE
cpu-demo-2   0/1     Pending   0          10s

查看有关 Pod 的详细信息，包含事件

由于节点上的 CPU 资源不足，无法调度容器

bash 复制代码

[root@k8s-1 pod]# kubectl describe pod cpu-demo-2 --namespace=cpu-example
Events:
  Type     Reason            Age               From               Message
  ----     ------            ----              ----               -------
  Warning  FailedScheduling  47s (x5 over 4m)  default-scheduler  0/3 nodes are available: 1 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate, 2 Insufficient cpu.

总结

Pod 的资源限制是保障集群稳定性、避免资源争抢的核心机制，通过定义 Pod 对 CPU、内存等计算资源的 "需求" 和 "上限"，实现资源的合理分配

requests -> Pod 运行所需的最小资源保障
limits -> Pod 能使用的最大资源上限