k8s error uploading crisocket处理过程

文章目录

• 问题描叙
• 处理方法
• 验证

问题描叙

join 一个worker节点时遇到erro uploading crisocket错误

root@k8sworker1:~# kubeadm join k8smaster.co.id:6443 --token t7gze7.edpwojsoac7t6fj5         --discovery-token-ca-cert-hash sha256:b0c2d8f49cd5eb4db816509a04c1d3ff82e468e71aef7c2837cf98ce52ad79b2 --v=5
I1119 16:16:57.124114   15072 join.go:419] [preflight] found NodeName empty; using OS hostname as NodeName
I1119 16:16:57.127983   15072 initconfiguration.go:123] detected and using CRI socket: unix:///var/run/containerd/containerd.sock
[preflight] Running pre-flight checks
I1119 16:16:57.129714   15072 preflight.go:93] [preflight] Running general checks
I1119 16:16:57.141874   15072 checks.go:278] validating the existence of file /etc/kubernetes/kubelet.conf
I1119 16:16:57.141958   15072 checks.go:278] validating the existence of file /etc/kubernetes/bootstrap-kubelet.conf
I1119 16:16:57.142126   15072 checks.go:102] validating the container runtime
I1119 16:16:57.143294   15072 checks.go:637] validating whether swap is enabled or not
I1119 16:16:57.143636   15072 checks.go:368] validating the presence of executable crictl
I1119 16:16:57.143656   15072 checks.go:368] validating the presence of executable conntrack
I1119 16:16:57.143667   15072 checks.go:368] validating the presence of executable ip
I1119 16:16:57.143678   15072 checks.go:368] validating the presence of executable iptables
I1119 16:16:57.143700   15072 checks.go:368] validating the presence of executable mount
I1119 16:16:57.143711   15072 checks.go:368] validating the presence of executable nsenter
I1119 16:16:57.143721   15072 checks.go:368] validating the presence of executable ethtool
I1119 16:16:57.143728   15072 checks.go:368] validating the presence of executable tc
I1119 16:16:57.143741   15072 checks.go:368] validating the presence of executable touch
I1119 16:16:57.143754   15072 checks.go:514] running all checks
I1119 16:16:57.164440   15072 checks.go:399] checking whether the given node name is valid and reachable using net.LookupHost
I1119 16:16:57.164837   15072 checks.go:603] validating kubelet version
I1119 16:16:57.244459   15072 checks.go:128] validating if the "kubelet" service is enabled and active
I1119 16:16:57.259143   15072 checks.go:201] validating availability of port 10250
I1119 16:16:57.259449   15072 checks.go:278] validating the existence of file /etc/kubernetes/pki/ca.crt
I1119 16:16:57.259550   15072 checks.go:428] validating if the connectivity type is via proxy or direct
I1119 16:16:57.259583   15072 join.go:538] [preflight] Discovering cluster-info
I1119 16:16:57.259607   15072 token.go:79] [discovery] Created cluster-info discovery client, requesting info from "k8smaster.pci.co.id:6443"
I1119 16:16:57.260061   15072 token.go:210] [discovery] Waiting for the cluster-info ConfigMap to receive a JWS signaturefor token ID "t7gze7"
I1119 16:16:57.276472   15072 token.go:117] [discovery] Requesting info from "k8smaster.pci.co.id:6443" again to validate TLS against the pinned public key
I1119 16:16:57.277403   15072 token.go:210] [discovery] Waiting for the cluster-info ConfigMap to receive a JWS signaturefor token ID "t7gze7"
I1119 16:16:57.291442   15072 token.go:134] [discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "k8smaster.pci.co.id:6443"
I1119 16:16:57.291592   15072 discovery.go:52] [discovery] Using provided TLSBootstrapToken as authentication credentials for the join process
I1119 16:16:57.291637   15072 join.go:552] [preflight] Fetching init configuration
I1119 16:16:57.291668   15072 join.go:598] [preflight] Retrieving KubeConfig objects
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
I1119 16:16:57.304891   15072 kubeproxy.go:55] attempting to download the KubeProxyConfiguration from ConfigMap "kube-proxy"
I1119 16:16:57.316062   15072 kubelet.go:73] attempting to download the KubeletConfiguration from ConfigMap "kubelet-config"
I1119 16:16:57.322185   15072 initconfiguration.go:115] skip CRI socket detection, fill with the default CRI socket unix:///var/run/containerd/containerd.sock
I1119 16:16:57.322544   15072 interface.go:432] Looking for default routes with IPv4 addresses
I1119 16:16:57.322587   15072 interface.go:437] Default route transits interface "enp0s3"
I1119 16:16:57.322672   15072 interface.go:209] Interface enp0s3 is up
I1119 16:16:57.322732   15072 interface.go:257] Interface "enp0s3" has 2 addresses :[172.19.6.8/23 fe80::4e72:b9ff:fe4f:ac9e/64].
I1119 16:16:57.322771   15072 interface.go:224] Checking addr  172.19.6.8/23.
I1119 16:16:57.322802   15072 interface.go:231] IP found 172.19.6.8
I1119 16:16:57.322835   15072 interface.go:263] Found valid IPv4 address 172.19.6.8 for interface "enp0s3".
I1119 16:16:57.322865   15072 interface.go:443] Found active IP 172.19.6.8
I1119 16:16:57.322919   15072 preflight.go:104] [preflight] Running configuration dependant checks
I1119 16:16:57.322953   15072 controlplaneprepare.go:225] [download-certs] Skipping certs download
I1119 16:16:57.323160   15072 kubelet.go:175] [kubelet-start] writing bootstrap kubelet config file at /etc/kubernetes/bootstrap-kubelet.conf
I1119 16:16:57.323630   15072 kubelet.go:190] [kubelet-start] writing CA certificate at /etc/kubernetes/pki/ca.crt
I1119 16:16:57.323740   15072 kubelet.go:206] [kubelet-start] Checking for an existing Node in the cluster with name "k8sworker1.pci.co.id" and status "Ready"
I1119 16:16:57.330379   15072 kubelet.go:221] [kubelet-start] Stopping the kubelet
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
[kubelet-check] The kubelet is healthy after 501.208975ms
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap
I1119 16:16:58.453645   15072 cert_rotation.go:140] Starting client certificate rotation controller
I1119 16:16:58.454706   15072 kubelet.go:318] [kubelet-start] preserving the crisocket information for the node
I1119 16:16:58.454800   15072 patchnode.go:31] [patchnode] Uploading the CRI Socket information "unix:///var/run/containerd/containerd.sock" to the Node API object "k8sworker1.pci.co.id" as an annotation
Unauthorized
error uploading crisocket
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/join.runKubeletWaitBootstrapPhase
        k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/join/kubelet.go:320

处理方法

1.检查hosts与hostname是否正确

sudo vi /etc/hostname 
sudo vi /etc/hosts

2.执行如下步骤

sudo kubeadm reset
rm -rf /var/lib/cni/
sudo rm -rf /var/lib/cni/

systemctl daemon-reload
systemctl restart kubelet
如果firewall已经disable这步可以省略
sudo iptables -F && sudo iptables -t nat -F && sudo iptables -t mangle -F && sudo iptables -X   

验证

1.再次join

 kubeadm join k8smaster.pci.co.id:6443 --token t7gze7.edpwojsoac7t6fj5         --discovery-token-ca-cert-hash sha256:b0c2d8f49cd5eb4db816509a04c1d3ff82e468e71aef7c2837cf98ce52ad79b2 --v=5
I1119 16:29:19.929004   15294 join.go:419] [preflight] found NodeName empty; using OS hostname as NodeName
I1119 16:29:19.931535   15294 initconfiguration.go:123] detected and using CRI socket: unix:///var/run/containerd/containerd.sock
[preflight] Running pre-flight checks
I1119 16:29:19.934208   15294 preflight.go:93] [preflight] Running general checks
I1119 16:29:19.943141   15294 checks.go:278] validating the existence of file /etc/kubernetes/kubelet.conf
I1119 16:29:19.943495   15294 checks.go:278] validating the existence of file /etc/kubernetes/bootstrap-kubelet.conf
I1119 16:29:19.943504   15294 checks.go:102] validating the container runtime
I1119 16:29:19.945810   15294 checks.go:637] validating whether swap is enabled or not
I1119 16:29:19.945938   15294 checks.go:368] validating the presence of executable crictl
I1119 16:29:19.945966   15294 checks.go:368] validating the presence of executable conntrack
I1119 16:29:19.945976   15294 checks.go:368] validating the presence of executable ip
I1119 16:29:19.945988   15294 checks.go:368] validating the presence of executable iptables
I1119 16:29:19.945999   15294 checks.go:368] validating the presence of executable mount
I1119 16:29:19.946096   15294 checks.go:368] validating the presence of executable nsenter
I1119 16:29:19.946108   15294 checks.go:368] validating the presence of executable ethtool
I1119 16:29:19.946116   15294 checks.go:368] validating the presence of executable tc
I1119 16:29:19.946124   15294 checks.go:368] validating the presence of executable touch
I1119 16:29:19.946135   15294 checks.go:514] running all checks
I1119 16:29:19.968353   15294 checks.go:399] checking whether the given node name is valid and reachable using net.LookupHost
I1119 16:29:19.968580   15294 checks.go:603] validating kubelet version
I1119 16:29:20.060023   15294 checks.go:128] validating if the "kubelet" service is enabled and active
I1119 16:29:20.081018   15294 checks.go:201] validating availability of port 10250
I1119 16:29:20.081247   15294 checks.go:278] validating the existence of file /etc/kubernetes/pki/ca.crt
I1119 16:29:20.081258   15294 checks.go:428] validating if the connectivity type is via proxy or direct
I1119 16:29:20.081277   15294 join.go:538] [preflight] Discovering cluster-info
I1119 16:29:20.081383   15294 token.go:79] [discovery] Created cluster-info discovery client, requesting info from "k8smaster.pci.co.id:6443"
I1119 16:29:20.082108   15294 token.go:210] [discovery] Waiting for the cluster-info ConfigMap to receive a JWS signaturefor token ID "t7gze7"
I1119 16:29:20.094460   15294 token.go:117] [discovery] Requesting info from "k8smaster.pci.co.id:6443" again to validate TLS against the pinned public key
I1119 16:29:20.096188   15294 token.go:210] [discovery] Waiting for the cluster-info ConfigMap to receive a JWS signaturefor token ID "t7gze7"
I1119 16:29:20.106197   15294 token.go:134] [discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "k8smaster.pci.co.id:6443"
I1119 16:29:20.106752   15294 discovery.go:52] [discovery] Using provided TLSBootstrapToken as authentication credentials for the join process
I1119 16:29:20.107122   15294 join.go:552] [preflight] Fetching init configuration
I1119 16:29:20.107267   15294 join.go:598] [preflight] Retrieving KubeConfig objects
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
I1119 16:29:20.119333   15294 kubeproxy.go:55] attempting to download the KubeProxyConfiguration from ConfigMap "kube-proxy"
I1119 16:29:20.133099   15294 kubelet.go:73] attempting to download the KubeletConfiguration from ConfigMap "kubelet-config"
I1119 16:29:20.139120   15294 initconfiguration.go:115] skip CRI socket detection, fill with the default CRI socket unix:///var/run/containerd/containerd.sock
I1119 16:29:20.139889   15294 interface.go:432] Looking for default routes with IPv4 addresses
I1119 16:29:20.140452   15294 interface.go:437] Default route transits interface "enp0s3"
I1119 16:29:20.140842   15294 interface.go:209] Interface enp0s3 is up
I1119 16:29:20.140952   15294 interface.go:257] Interface "enp0s3" has 2 addresses :[172.19.6.8/23 fe80::4e72:b9ff:fe4f:ac9e/64].
I1119 16:29:20.141144   15294 interface.go:224] Checking addr  172.19.6.8/23.
I1119 16:29:20.141225   15294 interface.go:231] IP found 172.19.6.8
I1119 16:29:20.141299   15294 interface.go:263] Found valid IPv4 address 172.19.6.8 for interface "enp0s3".
I1119 16:29:20.141940   15294 interface.go:443] Found active IP 172.19.6.8
I1119 16:29:20.142647   15294 preflight.go:104] [preflight] Running configuration dependant checks
I1119 16:29:20.142961   15294 controlplaneprepare.go:225] [download-certs] Skipping certs download
I1119 16:29:20.143512   15294 kubelet.go:175] [kubelet-start] writing bootstrap kubelet config file at /etc/kubernetes/bootstrap-kubelet.conf
I1119 16:29:20.144520   15294 kubelet.go:190] [kubelet-start] writing CA certificate at /etc/kubernetes/pki/ca.crt
I1119 16:29:20.144806   15294 kubelet.go:206] [kubelet-start] Checking for an existing Node in the cluster with name "k8sworker1.pci.co.id" and status "Ready"
I1119 16:29:20.148522   15294 kubelet.go:221] [kubelet-start] Stopping the kubelet
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
[kubelet-check] The kubelet is healthy after 1.002241182s
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap
I1119 16:29:21.742764   15294 kubelet.go:318] [kubelet-start] preserving the crisocket information for the node
I1119 16:29:21.742837   15294 patchnode.go:31] [patchnode] Uploading the CRI Socket information "unix:///var/run/containerd/containerd.sock" to the Node API object "k8sworker1.pci.co.id" as an annotation
I1119 16:29:21.743858   15294 cert_rotation.go:140] Starting client certificate rotation controller

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

2.验证

root@k8sworker1:~# kubectl get nodes
NAME                   STATUS     ROLES           AGE   VERSION
k8smaster.pci.co.id    NotReady   control-plane   23h   v1.31.2
k8sworker1.pci.co.id   Ready      <none>          10m   v1.31.2