Kubernetes 之 Pod 标签、节点选择器和节点亲和性

Kubernetes 之 Pod 标签、节点选择器和节点亲和性

标签的定义

标签其实就一对 key-value 键值对，被关联到资源对象上。在 Pod 中，标签可以标记该 Pod 的特形，例如该 Pod 的功能、版本等等以便于我们对 Pod 进行分组管理。标签可以在新建 Pod 的时候直接定义，也可以动态修改。每个 Pod 可以拥有多个标签，但是key值必须是唯一的。

标签的使用

# 查看 test 空间下所有 Pod 的标签
root@k8s-master1:~# kubectl get pods -n test --show-labels
NAME       READY   STATUS    RESTARTS   AGE   LABELS
k8s-test   1/1     Running   0          16s   app=k8s-test,feature=nginx

# 给 k8s-test 新增标签
root@k8s-master1:~# kubectl label -ntest pods/k8s-test version=1.0
pod/k8s-test labeled

# 查看 k8s-test Pod 的所有标签
root@k8s-master1:~# kubectl get pods/k8s-test -ntest --show-labels
NAME       READY   STATUS    RESTARTS   AGE   LABELS
k8s-test   1/1     Running   0          10m   app=k8s-test,feature=nginx,version=1.0

# 查看标签 version=1.0 的所有 Pod
root@k8s-master1:~# kubectl get pods -n test -l version=1.0
NAME       READY   STATUS    RESTARTS   AGE
k8s-test   1/1     Running   0          12m

# 查看具有 version 键的标签
root@k8s-master1:~# kubectl get pods -ntest -l version
NAME       READY   STATUS    RESTARTS   AGE
k8s-test   1/1     Running   0          13m
root@k8s-master1:~# kubectl get pods -ntest -L version
NAME       READY   STATUS    RESTARTS   AGE   VERSION
k8s-test   1/1     Running   0          14m   1.0

# 修改 version 键为 2.0
root@k8s-master1:~# kubectl label --overwrite -n test pods k8s-test version=2.0
pod/k8s-test labeled
root@k8s-master1:~# kubectl get pods k8s-test -L version -n test
NAME       READY   STATUS    RESTARTS   AGE   VERSION
k8s-test   1/1     Running   0          18m   2.0

# 移除 version 标签
root@k8s-master1:~# kubectl label pods -n test k8s-test version-
pod/k8s-test unlabeled
root@k8s-master1:~# kubectl get pods k8s-test --show-labels -n test
NAME       READY   STATUS    RESTARTS   AGE   LABELS
k8s-test   1/1     Running   0          22m   app=k8s-test,feature=nginx

节点选择器的定义

我们在创建 Pod 资源的时候，Pod 会根据 Scheduler 进行调度，默认会调度到随机的一个工作节点。如果我们想要将 Pod 调度到指定节点或者调度到一些具有相同特点的 Node 节点，可以使用 Pod 中的 nodeName 或者 nodeSelector 字段来指定要调度到的 Node 节点。

节点选择器的使用

1. 配置nodeName在 k8s-master1 节点上。原则上控制节点不分配业务 Pod ，但是nodeName优先级很高，可能将其指定到 控制节点上。

   apiVersion: v1
   kind: Pod
   metadata:
     name: k8s-test
     namespace: test
     labels:
       app: k8s-test
       feature: nginx
   spec:
     nodeName: k8s-master1
     containers:
       - name:  k8s-test
         ports:
           - containerPort: 80
         image: k8s-test:v1.0
         imagePullPolicy: IfNotPresent
         resources:
           requests:
             memory: "1Gi"
             cpu: "1"
           limits:
             memory: "2Gi"
             cpu: "2"

   # 查看 Pod
   root@k8s-master1:~# kubectl get pods -owide -ntest
   NAME       READY   STATUS    RESTARTS   AGE   IP               NODE          NOMINATED NODE   READINESS GATES
   k8s-test   1/1     Running   0          90s   10.244.159.137   k8s-master1   <none>           <none>

2. 给 k8s-worker2 节点创建 os=debian和env=test两个标签

   root@k8s-master1:~# kubectl label nodes k8s-worker2 os=debian env=test -ntest
   node/k8s-worker2 labeled
   root@k8s-master1:~# kubectl get nodes k8s-worker2 --show-labels
   NAME          STATUS   ROLES    AGE    VERSION   LABELS
   k8s-worker2   Ready    <none>   3d4h   v1.29.4   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,env=test,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-worker2,kubernetes.io/os=linux,os=debian

3. 测试nodeName和nodeSelector并存的情况

   apiVersion: v1
   kind: Pod
   metadata:
     name: k8s-test
     namespace: test
     labels:
       app: k8s-test
       feature: nginx
   spec:
     nodeName: k8s-master1
     nodeSelector:
       os: debian
       env: test
     containers:
       - name:  k8s-test
         ports:
           - containerPort: 80
         image: k8s-test:v1.0
         imagePullPolicy: IfNotPresent
         resources:
           requests:
             memory: "1Gi"
             cpu: "1"
           limits:
             memory: "2Gi"
             cpu: "2"

   root@k8s-master1:~# kubectl describe pods k8s-test -ntest
   Name:             k8s-test
   Namespace:        test
   Priority:         0
   Service Account:  default
   Node:             k8s-master1/192.168.31.60
   Start Time:       Thu, 16 May 2024 13:04:07 +0800
   Labels:           app=k8s-test
                     feature=nginx
   Annotations:      cni.projectcalico.org/podIP:
                     cni.projectcalico.org/podIPs:
   Status:           Failed
   Reason:           NodeAffinity
   Message:          Pod was rejected: Predicate NodeAffinity failed
   
   # 这里说明条件冲突，导致创建失败。将 nodeName 修改为 k8s-worker2 后成功
   root@k8s-master1:~# kubectl get pods -ntest
   NAME       READY   STATUS    RESTARTS   AGE
   k8s-test   1/1     Running   0          6s
   
   # 将 nodeName 去除，将 nodeSelect 中 os=windows
   root@k8s-master1:~# kubectl describe pods k8s-test -ntest
   Events:
     Type     Reason            Age   From               Message
     ----     ------            ----  ----               -------
     Warning  FailedScheduling  22s   default-scheduler  0/3 nodes are available: 1 node(s) had untolerated taint {node-role.kubernetes.io/control-plane: }, 2 node(s) didn't match Pod's node affinity/selector. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling.
     
   # 综上可见,所有条件均是与的关系

   节点亲和性的定义

   节点亲和性是 Pod 在调度过程中一系列调度约束规则，主要针对节点选择器的匹配，匹配度越高，则该节点执行该 Pod 的概率就越高。节点亲和性分为硬亲和性requiredDuringSchedulingIgnoredDuringExecution和软亲和性preferredDuringSchedulin $Data2012

   gIgnoredDuringExecution，硬亲和性代表必须满足，软亲和性在不满足约束规则的条件上，仍可以随机启动。

   硬亲和性使用

   apiVersion: v1
   kind: Pod
   metadata:
     name: node-hard-affinity
     namespace: default
     labels:
       app: k8s-test
   spec:
     affinity:
       nodeAffinity:
         requiredDuringSchedulingIgnoredDuringExecution:
           nodeSelectorTerms:
           - matchExpressions:
             - key: role
               operator: In
               values:
               - worker
     containers:
     - name: k8s-test
       image: k8s-test:v1.0
       imagePullPolicy: IfNotPresent
       ports:
       - containerPort: 80

   root@k8s-master1:~# kubectl get node -L role
   NAME          STATUS   ROLES           AGE    VERSION   ROLE
   k8s-master1   Ready    control-plane   4d7h   v1.29.4
   k8s-worker1   Ready    <none>          4d7h   v1.29.4
   k8s-worker2   Ready    <none>          4d7h   v1.29.4   worker
   root@k8s-master1:~# kubectl get pods -owide
   NAME                 READY   STATUS    RESTARTS   AGE    IP              NODE          NOMINATED NODE   READINESS GATES
   node-hard-affinity   1/1     Running   0          2m5s   10.244.126.12   k8s-worker2   <none>           <none>

   软亲和性使用

   metadata:
     name: node-soft-affinity
     namespace: default
     labels:
       app: k8s-test
   spec:
     affinity:
       nodeAffinity:
         preferredDuringSchedulingIgnoredDuringExecution:
         - preference:
             matchExpressions:
             - key: memory
               operator: Lt
               values:
               - "2048"
           weight: 20 # 权重，区间在1-100，谁权重高就选谁
         - preference:
             matchExpressions:
             - key: memory
               operator: Gt
               values:
               - "2048"
           weight: 10
     containers:
     - name: k8s-test
       image: k8s-test:v1.0
       imagePullPolicy: IfNotPresent
       ports:
       - containerPort: 80

   # 本 Pod 亲和内存大于 2048 和内存小于 2048 的两种情况，但小于 2048 的节点权重更大
   root@k8s-master1:~# kubectl get nodes -L memory
   NAME          STATUS   ROLES           AGE     VERSION   MEMORY
   k8s-master1   Ready    control-plane   4d13h   v1.29.4
   k8s-worker1   Ready    <none>          4d12h   v1.29.4   1024
   k8s-worker2   Ready    <none>          4d12h   v1.29.4   4096
   root@k8s-master1:~# kubectl get pods -owide
   NAME                 READY   STATUS    RESTARTS   AGE    IP              NODE          NOMINATED NODE   READINESS GATES
   node-soft-affinity   1/1     Running   0          2m7s   10.244.194.80   k8s-worker1   <none>           <none>