K8s 单机二进制部署步骤(复制粘贴即可)

@土豆2026-03-19 8:11

K8s v1.28.7 单Master节点纯二进制部署文档

文档说明

  • 本文档适用于单机部署场景，部署前只需直接替换主机名（master01）和主机IP（192.168.133.140），替换完成后，跟随文档步骤复制粘贴命令即可完成部署。

  • 本文档实现K8s v1.28.7 单Master节点纯二进制部署，kube-proxy提供二进制部署和DaemonSet守护进程部署两种方式，无语法错误、无多余依赖，所有命令可直接复制执行。

  • 适配系统：CentOS 7（Linux x86_64）

核心说明

  1. 部署全程无kubeadm依赖，手动生成CA及所有组件证书，部署kube-apiserver后再生成admin.conf（确保api启动后可正常使用）；各组件对应conf文件均在组件部署后生成，kube-proxy两种部署方式相互独立，可根据实际需求选择。同时保证kubelet能自动注册到集群。
  2. etcd 组件和kubernetes组件共用一套CA证书(front-proxy除外)，且所有证书统一保存在/etc/kubernetes/pki目录下(kubelet和kube-proxy的除外)。
  3. kubelet和kube-proxy的相关文件分别保存在 /var/lib/kubelet 和 /var/lib/kube-proxy (当容器部署kube-proxy除外)。

一、环境准备

1.1 系统基础配置

首先安装必要依赖并升级内核，确保系统满足K8s部署基础要求：

bash 复制代码 
# 1. 必要的依赖 （libseccomp libseccomp-devel 是 runc 必要的依赖）
yum install -y  ipvsadm ipset conntrack-tools socat ebtables  libseccomp libseccomp-devel
# 内核升级  
wget https://dl.lamp.sh/kernel/el7/kernel-ml-6.7.8-1.el7.elrepo.x86_64.rpm  # 可直接复制到浏览器下载
rpm -ivh kernel-ml-6.7.8-1.el7.elrepo.x86_64.rpm
grub2-set-default 0  # 升级内核并设置内核启动顺序，重启生效
reboot

# 可选
yum install -y wget vim net-tools iftop telnet  bash-completion.noarch

配置K8s基础内核参数，关闭防火墙、SELinux和swap，加载ipvs相关模块：

bash 复制代码 
# 2. 简单 k8s环境 内核参数版
systemctl stop firewalld && systemctl disable firewalld
setenforce 0 && sed -i 's/^SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
swapoff -a && sed -i '/swap/s/^/#/' /etc/fstab
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
vm.swappiness = 0
EOF
sysctl --system

cat > /etc/sysconfig/modules/ipvs.modules << EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack
EOF
chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules
lsmod | grep -E "ip_vs|nf_conntrack"

生产环境优化版系统初始化脚本（推荐生产环境使用，包含更全面的内核、文件描述符等优化配置）：

bash 复制代码 
# 3. 生产 优化 版
# cat initOS.sh
#!/usr/bin/env bash

swapoff -a
sed -i /^[^#]*swap*/s/^/\#/g /etc/fstab

# See https://github.com/kubernetes/website/issues/14457
if [ -f /etc/selinux/config ]; then
  sed -ri 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
fi
# for ubuntu: sudo apt install selinux-utils
# for centos: yum install selinux-policy
if command -v setenforce &> /dev/null
then
  setenforce 0
  getenforce
fi

cat >> /etc/sysctl.conf <<-EOF
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_local_reserved_ports = 30000-32767
net.core.netdev_max_backlog = 65535
net.core.rmem_max = 33554432
net.core.wmem_max = 33554432
net.core.somaxconn = 32768
net.ipv4.tcp_max_syn_backlog = 1048576
net.ipv4.neigh.default.gc_thresh1 = 512
net.ipv4.neigh.default.gc_thresh2 = 2048
net.ipv4.neigh.default.gc_thresh3 = 4096
net.ipv4.tcp_retries2 = 15
net.ipv4.tcp_max_tw_buckets = 1048576
net.ipv4.tcp_max_orphans = 65535
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_intvl = 30
net.ipv4.tcp_keepalive_probes = 10
net.ipv4.udp_rmem_min = 131072
net.ipv4.udp_wmem_min = 131072
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.all.arp_accept = 1
net.ipv4.conf.default.arp_accept = 1
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.default.arp_ignore = 1
vm.max_map_count = 262144
vm.swappiness = 0
vm.overcommit_memory = 0
fs.inotify.max_user_instances = 524288
fs.inotify.max_user_watches = 524288
fs.pipe-max-size = 4194304
fs.aio-max-nr = 262144
kernel.pid_max = 65535
kernel.watchdog_thresh = 5
kernel.hung_task_timeout_secs = 5
EOF

#See https://help.aliyun.com/document_detail/118806.html#uicontrol-e50-ddj-w0y
sed -r -i "s@#{0,}?net.ipv4.tcp_tw_recycle ?= ?(0|1|2)@net.ipv4.tcp_tw_recycle = 0@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.ipv4.tcp_tw_reuse ?= ?(0|1)@net.ipv4.tcp_tw_reuse = 0@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.ipv4.conf.all.rp_filter ?= ?(0|1|2)@net.ipv4.conf.all.rp_filter = 0@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.ipv4.conf.default.rp_filter ?= ?(0|1|2)@net.ipv4.conf.default.rp_filter = 0@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.ipv4.ip_forward ?= ?(0|1)@net.ipv4.ip_forward = 1@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.bridge.bridge-nf-call-arptables ?= ?(0|1)@net.bridge.bridge-nf-call-arptables = 1@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.bridge.bridge-nf-call-ip6tables ?= ?(0|1)@net.bridge.bridge-nf-call-ip6tables = 1@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.bridge.bridge-nf-call-iptables ?= ?(0|1)@net.bridge.bridge-nf-call-iptables = 1@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.ipv4.ip_local_reserved_ports ?= ?([0-9]{1,}-{0,1},{0,1}){1,}@net.ipv4.ip_local_reserved_ports = 30000-32767@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?vm.max_map_count ?= ?([0-9]{1,})@vm.max_map_count = 262144@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?vm.swappiness ?= ?([0-9]{1,})@vm.swappiness = 0@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?fs.inotify.max_user_instances ?= ?([0-9]{1,})@fs.inotify.max_user_instances = 524288@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?kernel.pid_max ?= ?([0-9]{1,})@kernel.pid_max = 65535@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?vm.overcommit_memory ?= ?(0|1|2)@vm.overcommit_memory = 0@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?fs.inotify.max_user_watches ?= ?([0-9]{1,})@fs.inotify.max_user_watches = 524288@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?fs.pipe-max-size ?= ?([0-9]{1,})@fs.pipe-max-size = 4194304@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.core.netdev_max_backlog ?= ?([0-9]{1,})@net.core.netdev_max_backlog = 65535@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.core.rmem_max ?= ?([0-9]{1,})@net.core.rmem_max = 33554432@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.core.wmem_max ?= ?([0-9]{1,})@net.core.wmem_max = 33554432@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.ipv4.tcp_max_syn_backlog ?= ?([0-9]{1,})@net.ipv4.tcp_max_syn_backlog = 1048576@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.ipv4.neigh.default.gc_thresh1 ?= ?([0-9]{1,})@net.ipv4.neigh.default.gc_thresh1 = 512@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.ipv4.neigh.default.gc_thresh2 ?= ?([0-9]{1,})@net.ipv4.neigh.default.gc_thresh2 = 2048@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.ipv4.neigh.default.gc_thresh3 ?= ?([0-9]{1,})@net.ipv4.neigh.default.gc_thresh3 = 4096@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.core.somaxconn ?= ?([0-9]{1,})@net.core.somaxconn = 32768@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.ipv4.conf.eth0.arp_accept ?= ?(0|1)@net.ipv4.conf.eth0.arp_accept = 1@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?fs.aio-max-nr ?= ?([0-9]{1,})@fs.aio-max-nr = 262144@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.ipv4.tcp_retries2 ?= ?([0-9]{1,})@net.ipv4.tcp_retries2 = 15@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.ipv4.tcp_max_tw_buckets ?= ?([0-9]{1,})@net.ipv4.tcp_max_tw_buckets = 1048576@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.ipv4.tcp_max_orphans ?= ?([0-9]{1,})@net.ipv4.tcp_max_orphans = 65535@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.ipv4.tcp_keepalive_time ?= ?([0-9]{1,})@net.ipv4.tcp_keepalive_time = 600@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.ipv4.tcp_keepalive_intvl ?= ?([0-9]{1,})@net.ipv4.tcp_keepalive_intvl = 30@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.ipv4.tcp_keepalive_probes ?= ?([0-9]{1,})@net.ipv4.tcp_keepalive_probes = 10@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.ipv4.udp_rmem_min ?= ?([0-9]{1,})@net.ipv4.udp_rmem_min = 131072@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.ipv4.udp_wmem_min ?= ?([0-9]{1,})@net.ipv4.udp_wmem_min = 131072@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.ipv4.conf.all.arp_ignore ?= ??(0|1|2)@net.ipv4.conf.all.arp_ignore = 1@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?net.ipv4.conf.default.arp_ignore ?= ??(0|1|2)@net.ipv4.conf.default.arp_ignore = 1@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?kernel.watchdog_thresh ?= ?([0-9]{1,})@kernel.watchdog_thresh = 5@g" /etc/sysctl.conf
sed -r -i "s@#{0,}?kernel.hung_task_timeout_secs ?= ?([0-9]{1,})@kernel.hung_task_timeout_secs = 5@g" /etc/sysctl.conf

tmpfile="$$.tmp"
awk ' !x[$0]++{print > "'$tmpfile'"}' /etc/sysctl.conf
mv $tmpfile /etc/sysctl.conf

# ulimit

cat >> /etc/security/limits.conf <<-EOF
* soft nofile 1048576
* hard nofile 1048576
* soft nproc 65536
* hard nproc 65536
* soft memlock unlimited
* hard memlock unlimited
EOF

sed -r -i  "s@#{0,}?\* soft nofile ?([0-9]{1,})@\* soft nofile 1048576@g" /etc/security/limits.conf
sed -r -i  "s@#{0,}?\* hard nofile ?([0-9]{1,})@\* hard nofile 1048576@g" /etc/security/limits.conf
sed -r -i  "s@#{0,}?\* soft nproc ?([0-9]{1,})@\* soft nproc 65536@g" /etc/security/limits.conf
sed -r -i  "s@#{0,}?\* hard nproc ?([0-9]{1,})@\* hard nproc 65536@g" /etc/security/limits.conf
sed -r -i  "s@#{0,}?\* soft memlock ?([0-9]{1,}([TGKM]B){0,1}|unlimited)@\* soft memlock unlimited@g" /etc/security/limits.conf
sed -r -i  "s@#{0,}?\* hard memlock ?([0-9]{1,}([TGKM]B){0,1}|unlimited)@\* hard memlock unlimited@g" /etc/security/limits.conf

tmpfile="$$.tmp"
awk ' !x[$0]++{print > "'$tmpfile'"}' /etc/security/limits.conf
mv $tmpfile /etc/security/limits.conf

# Check if firewalld service exists and is running
systemctl status firewalld 1>/dev/null 2>/dev/null
if [ $? -eq 0 ]; then
    # Firewall service exists and is running, stop and disable it
    systemctl stop firewalld 1>/dev/null 2>/dev/null
    systemctl disable firewalld 1>/dev/null 2>/dev/null
fi
# Check if ufw service exists and is running
systemctl status ufw 1>/dev/null 2>/dev/null
if [ $? -eq 0 ]; then
    # ufw service exists and is running, stop and disable it
    systemctl stop ufw 1>/dev/null 2>/dev/null
    systemctl disable ufw 1>/dev/null 2>/dev/null
fi

modinfo br_netfilter > /dev/null 2>&1
if [ $? -eq 0 ]; then
   modprobe br_netfilter
   mkdir -p /etc/modules-load.d
   echo 'br_netfilter' > /etc/modules-load.d/kubekey-br_netfilter.conf
fi

modinfo overlay > /dev/null 2>&1
if [ $? -eq 0 ]; then
   modprobe overlay
   echo 'overlay' >> /etc/modules-load.d/kubekey-br_netfilter.conf
fi

modprobe ip_vs
modprobe ip_vs_rr
modprobe ip_vs_wrr
modprobe ip_vs_sh

cat > /etc/modules-load.d/kube_proxy-ipvs.conf << EOF
ip_vs
ip_vs_rr
ip_vs_wrr
ip_vs_sh
EOF

modprobe nf_conntrack_ipv4 1>/dev/null 2>/dev/null
if [ $? -eq 0 ]; then
   echo 'nf_conntrack_ipv4' > /etc/modules-load.d/kube_proxy-ipvs.conf
else
   modprobe nf_conntrack
   echo 'nf_conntrack' > /etc/modules-load.d/kube_proxy-ipvs.conf
fi
sysctl -p

sync
# echo 3 > /proc/sys/vm/drop_caches

# Make sure the iptables utility doesn't use the nftables backend.
update-alternatives --set iptables /usr/sbin/iptables-legacy >/dev/null 2>&1 || true
update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy >/dev/null 2>&1 || true
update-alternatives --set arptables /usr/sbin/arptables-legacy >/dev/null 2>&1 || true
update-alternatives --set ebtables /usr/sbin/ebtables-legacy >/dev/null 2>&1 || true

执行上述生产环境优化脚本，完成系统初始化：

bash 复制代码 
# 执行内核初始化
sh initOS.sh

1.2 安装cfssl证书生成工具

后续生成CA及各组件证书需依赖cfssl工具，执行以下命令安装并验证：

bash 复制代码 
wget https://github.com/cloudflare/cfssl/releases/download/v1.6.4/cfssl_1.6.4_linux_amd64 -O /usr/local/bin/cfssl
wget https://github.com/cloudflare/cfssl/releases/download/v1.6.4/cfssljson_1.6.4_linux_amd64 -O /usr/local/bin/cfssljson
chmod +x /usr/local/bin/cfssl /usr/local/bin/cfssljson
cfssl version

1.3 下载kubernetes软件

下载K8s v1.28.7版本二进制文件，并复制到系统可执行目录：

bash 复制代码 
K8S_VERSION=v1.28.7
wget https://dl.k8s.io/${K8S_VERSION}/kubernetes-server-linux-amd64.tar.gz
tar zxf kubernetes-server-linux-amd64.tar.gz
# 一次性复制所有K8s可执行文件到/usr/local/bin，无需单独复制单个组件
cp kubernetes/server/bin/* /usr/local/bin/

1.4 下载etcd软件

下载etcd v3.5.10版本（与K8s v1.28.7适配），并复制可执行文件到系统目录：

bash 复制代码 
wget https://github.com/etcd-io/etcd/releases/download/v3.5.10/etcd-v3.5.10-linux-amd64.tar.gz
tar zxf etcd-v3.5.10-linux-amd64.tar.gz
cp etcd-v3.5.10-linux-amd64/etcd* /usr/local/bin/

1.5 下载containerd软件

下载containerd v1.7.11版本（作为K8s容器运行时），并解压到指定目录：

bash 复制代码 
wget https://github.com/containerd/containerd/releases/download/v1.7.11/containerd-1.7.11-linux-amd64.tar.gz
tar zxf containerd-1.7.11-linux-amd64.tar.gz -C /usr/local/

二、生成CA证书及所有组件证书（除kubelet、kube-proxy）

创建证书存放目录，生成CA基础配置及各类组件证书，所有证书统一存放于/etc/kubernetes/pki目录：

bash 复制代码 
mkdir -p /etc/kubernetes/pki /opt/cert
cd /opt/cert
# 1. ETCD 和 k8s组件共用的CA
cat > ca-config.json << EOF
{
  "signing": {
    "default": {
      "expiry": "876000h"
    },
    "profiles": {
      "kubernetes": {
        "usages": ["signing", "key encipherment", "server auth", "client auth"],
        "expiry": "876000h"
      }
    }
  }
}
EOF

cat > ca-csr.json << EOF
{
  "CN": "kubernetes",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "Beijing",
      "L": "Beijing",
      "O": "Kubernetes",
      "OU": "CA"
    }
  ]
}
EOF

cfssl gencert -initca ca-csr.json | cfssljson -bare /opt/cert/ca

mkdir -p  /etc/kubernetes/pki/
mv /opt/cert/ca*.pem /etc/kubernetes/pki/

cat > /opt/cert/apiserver-csr.json << EOF
{
  "CN": "kubernetes",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "Beijing",
      "L": "Beijing",
      "O": "Kubernetes",
      "OU": "API Server"
    }
  ],
  "hosts": [
    "127.0.0.1",
    "192.168.133.140",
    "10.96.0.1",
    "kubernetes",
    "kubernetes.default",
    "kubernetes.default.svc",
    "kubernetes.default.svc.cluster",
    "kubernetes.default.svc.cluster.local"
  ]
}
EOF

cfssl gencert -ca=/etc/kubernetes/pki/ca.pem \
  -ca-key=/etc/kubernetes/pki/ca-key.pem \
  -config=/opt/cert/ca-config.json \
  -profile=kubernetes /opt/cert/apiserver-csr.json | cfssljson -bare /opt/cert/apiserver
mv /opt/cert/apiserver*.pem /etc/kubernetes/pki/

# 2. kube-apiserver 访问 kubelet用的证书
cat > /opt/cert/apiserver-kubelet-client-csr.json << EOF
{
  "CN": "apiserver-kubelet-client",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "Beijing",
      "L": "Beijing",
      "O": "system:masters",
      "OU": "Kubernetes"
    }
  ]
}
EOF

cfssl gencert -ca=/etc/kubernetes/pki/ca.pem \
  -ca-key=/etc/kubernetes/pki/ca-key.pem \
  -config=/opt/cert/ca-config.json \
  -profile=kubernetes /opt/cert/apiserver-kubelet-client-csr.json | cfssljson -bare /opt/cert/apiserver-kubelet-client
  
mv /opt/cert/apiserver-kubelet-client*.pem /etc/kubernetes/pki/

# 3. 生成front-proxy-ca证书（前端代理用，单独的ca）
cat > /opt/cert/front-proxy-ca-csr.json << EOF
{
  "CN": "front-proxy-ca",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "Beijing",
      "L": "Beijing",
      "O": "Kubernetes",
      "OU": "Front Proxy CA"
    }
  ]
}
EOF

cfssl gencert -initca /opt/cert/front-proxy-ca-csr.json | cfssljson -bare /opt/cert/front-proxy-ca

mv /opt/cert/front-proxy-ca*.pem /etc/kubernetes/pki/

# 生成front-proxy-client证书（前端代理客户端用）
cat > /opt/cert/front-proxy-client-csr.json << EOF
{
  "CN": "front-proxy-client",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "Beijing",
      "L": "Beijing",
      "O": "Kubernetes",
      "OU": "Front Proxy Client"
    }
  ]
}
EOF

cfssl gencert -ca=/etc/kubernetes/pki/front-proxy-ca.pem \
  -ca-key=/etc/kubernetes/pki/front-proxy-ca-key.pem \
  -config=/opt/cert/ca-config.json \
  -profile=kubernetes /opt/cert/front-proxy-client-csr.json | cfssljson -bare /opt/cert/front-proxy-client
  
mv /opt/cert/front-proxy-client*.pem /etc/kubernetes/pki/


# 4. admin 证书（集群管理员用,如: kubectl 命令行）
cat > /opt/cert/admin-csr.json << EOF
{
  "CN": "admin",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "Beijing",
      "L": "Beijing",
      "O": "system:masters",
      "OU": "Kubernetes"
    }
  ]
}
EOF

cfssl gencert -ca=/etc/kubernetes/pki/ca.pem \
  -ca-key=/etc/kubernetes/pki/ca-key.pem \
  -config=/opt/cert/ca-config.json \
  -profile=kubernetes /opt/cert/admin-csr.json | cfssljson -bare /opt/cert/admin
  
mv /opt/cert/admin*.pem /etc/kubernetes/pki/

# 5. 生成service account密钥（sa相关）
openssl genrsa -out /etc/kubernetes/pki/sa.key 2048
openssl rsa -in /etc/kubernetes/pki/sa.key -pubout -out /etc/kubernetes/pki/sa.pub

三、部署etcd

etcd作为K8s集群的数据存储组件，需先生成其专属证书，再配置系统服务并启动，最后验证健康状态：

bash 复制代码 
mkdir -p /var/lib/etcd

# 1. etcd-member证书
cat > /opt/cert/etcd-member-csr.json << EOF
{
  "CN": "etcd-member",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "Beijing",
      "L": "Beijing",
      "O": "Kubernetes",
      "OU": "etcd"
    }
  ],
  "hosts": [
    "127.0.0.1",
    "192.168.133.140"
  ]
}
EOF

cfssl gencert -ca=/etc/kubernetes/pki/ca.pem \
  -ca-key=/etc/kubernetes/pki/ca-key.pem \
  -config=/opt/cert/ca-config.json \
  -profile=kubernetes /opt/cert/etcd-member-csr.json | cfssljson -bare /opt/cert/etcd-member
mv /opt/cert/etcd-member*.pem /etc/kubernetes/pki/

# 2. etcd.env, etcd 配置参数
cat > /etc/etcd.env << EOF
# Environment file for etcd v3.5.13
ETCD_DATA_DIR=/var/lib/etcd
ETCD_ADVERTISE_CLIENT_URLS=https://192.168.133.140:2379
ETCD_INITIAL_ADVERTISE_PEER_URLS=https://192.168.133.140:2380
ETCD_INITIAL_CLUSTER_STATE=new
ETCD_METRICS=basic
ETCD_LISTEN_CLIENT_URLS=https://192.168.133.140:2379,https://127.0.0.1:2379
ETCD_INITIAL_CLUSTER_TOKEN=k8s_etcd
ETCD_LISTEN_PEER_URLS=https://192.168.133.140:2380
ETCD_NAME=etcd-master01
ETCD_PROXY=off
ETCD_ENABLE_V2=true
ETCD_INITIAL_CLUSTER=etcd-master01=https://192.168.133.140:2380
ETCD_ELECTION_TIMEOUT=5000
ETCD_HEARTBEAT_INTERVAL=250
ETCD_AUTO_COMPACTION_RETENTION=8
ETCD_SNAPSHOT_COUNT=10000

# TLS settings
ETCD_TRUSTED_CA_FILE=/etc/kubernetes/pki/ca.pem
ETCD_CERT_FILE=/etc/kubernetes/pki/etcd-member.pem
ETCD_KEY_FILE=/etc/kubernetes/pki/etcd-member-key.pem
ETCD_CLIENT_CERT_AUTH=true

ETCD_PEER_TRUSTED_CA_FILE=/etc/kubernetes/pki/ca.pem
ETCD_PEER_CERT_FILE=/etc/kubernetes/pki/etcd-member.pem
ETCD_PEER_KEY_FILE=/etc/kubernetes/pki/etcd-member-key.pem
ETCD_PEER_CLIENT_CERT_AUTH=true

# CLI settings
ETCDCTL_ENDPOINTS=https://127.0.0.1:2379
ETCDCTL_CACERT=/etc/kubernetes/pki/ca.pem
ETCDCTL_KEY=/etc/kubernetes/pki/etcd-member-key.pem
ETCDCTL_CERT=/etc/kubernetes/pki/etcd-member.pem
EOF

# 3. etcd 启动服务
cat > /etc/systemd/system/etcd.service << EOF
[Unit]
Description=etcd
After=network.target

[Service]
User=root
Type=notify
Nice=-20
OOMScoreAdjust=-1000
EnvironmentFile=-/etc/etcd.env
ExecStart=/usr/local/bin/etcd
NotifyAccess=all
RestartSec=10s
LimitNOFILE=40000
Restart=always

[Install]
WantedBy=multi-user.target
EOF

# 4. 启动 etcd 并验证
systemctl daemon-reload && systemctl enable etcd --now
systemctl status etcd
etcdctl --endpoints=https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/ca.pem --cert=/etc/kubernetes/pki/etcd-member.pem --key=/etc/kubernetes/pki/etcd-member-key.pem endpoint health

四、部署容器运行时（containerd）

配置containerd系统服务，修改默认配置（适配systemd cgroup和国内镜像），启动并验证服务状态：

bash 复制代码 
# 1. containerd 启动服务
cat > /etc/systemd/system/containerd.service << EOF
[Unit]
Description=containerd container runtime
Documentation=https://containerd.io
After=network.target local-fs.target

[Service]
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/containerd

Type=notify
Delegate=yes
KillMode=process
Restart=always
RestartSec=5
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
LimitNOFILE=1048576
LimitMEMLOCK=infinity
# Comment TasksMax if your systemd version does not supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
OOMScoreAdjust=-999

[Install]
WantedBy=multi-user.target
EOF

# 2. containerd 配置文件
mkdir -p /etc/containerd
containerd config default > /etc/containerd/config.toml
sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/' /etc/containerd/config.toml
sed -i '/sandbox_image/c\    sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.9"' /etc/containerd/config.toml
grep sandbox_image /etc/containerd/config.toml

# 3. 启动 containerd 服务
systemctl daemon-reload && systemctl enable containerd 
systemctl restart containerd
systemctl status containerd

五、部署Master核心组件（kube-apiserver、kube-controller-manager、kube-scheduler）

说明：kube-controller-manager 和 kube-scheduler 组件需开启以下启动参数，确保Prometheus可无需认证获取监控指标：

bash 复制代码 
# kube-controller-manager 和 kube-scheduler 组件开启以下启动参数，Prometheus就不需要认证就可以获取监控指标
--authorization-always-allow-paths=/metrics

5.1 部署kube-apiserver

kube-apiserver是K8s集群的核心入口，配置系统服务、启动服务后，生成admin.conf配置文件，确保集群可正常访问：

bash 复制代码 
# 1. token.csv
cat > /etc/kubernetes/token.csv << EOF
ed0248a55cde31f039f9c9019c3a5440,kubelet-bootstrap,10001,"system:node-bootstrapper"
EOF

# 2. kube-apiserver
cat > /etc/systemd/system/kube-apiserver.service << EOF
[Unit]
Description=Kubernetes API Server
After=network.target etcd.service

[Service]
ExecStart=/usr/local/bin/kube-apiserver \
  --advertise-address=192.168.133.140 \
  --allow-privileged=true \
  --authorization-mode=Node,RBAC \
  --bind-address=0.0.0.0 \
  --client-ca-file=/etc/kubernetes/pki/ca.pem \
  --enable-admission-plugins=NamespaceLifecycle,NodeRestriction,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota \
  --enable-bootstrap-token-auth=true \
  --etcd-cafile=/etc/kubernetes/pki/ca.pem \
  --etcd-certfile=/etc/kubernetes/pki/etcd-member.pem \
  --etcd-keyfile=/etc/kubernetes/pki/etcd-member-key.pem \
  --etcd-servers=https://192.168.133.140:2379 \
  --feature-gates=RotateKubeletServerCertificate=true \
  --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.pem \
  --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client-key.pem \
  --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname \
  --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.pem \
  --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client-key.pem \
  --requestheader-allowed-names=front-proxy-client \
  --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.pem \
  --requestheader-extra-headers-prefix=X-Remote-Extra- \
  --requestheader-group-headers=X-Remote-Group \
  --requestheader-username-headers=X-Remote-User \
  --secure-port=6443 \
  --service-account-issuer=https://kubernetes.default.svc.cluster.local \
  --service-account-key-file=/etc/kubernetes/pki/sa.pub \
  --service-account-signing-key-file=/etc/kubernetes/pki/sa.key \
  --service-cluster-ip-range=10.96.0.0/16 \
  --tls-cert-file=/etc/kubernetes/pki/apiserver.pem \
  --tls-private-key-file=/etc/kubernetes/pki/apiserver-key.pem \
  --token-auth-file=/etc/kubernetes/token.csv \
  --v=2
Restart=always
RestartSec=5
LimitNOFILE=65535

[Install]
WantedBy=multi-user.target
EOF

# 3. 启动 kube-apiserver
systemctl daemon-reload && systemctl enable kube-apiserver --now
systemctl status kube-apiserver

# 4. admin.conf
MASTER_IP=192.168.133.140
kubectl config set-cluster kubernetes \
  --certificate-authority=/etc/kubernetes/pki/ca.pem \
  --embed-certs=true \
  --server=https://${MASTER_IP}:6443 \
  --kubeconfig=/etc/kubernetes/admin.conf
kubectl config set-credentials admin \
  --client-certificate=/etc/kubernetes/pki/admin.pem \
  --client-key=/etc/kubernetes/pki/admin-key.pem \
  --embed-certs=true \
  --kubeconfig=/etc/kubernetes/admin.conf
kubectl config set-context kubernetes \
  --cluster=kubernetes \
  --user=admin \
  --kubeconfig=/etc/kubernetes/admin.conf
kubectl config use-context kubernetes --kubeconfig=/etc/kubernetes/admin.conf

mkdir -p ~/.kube
cp /etc/kubernetes/admin.conf ~/.kube/config

kubectl get cs,nodes

# echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> /etc/profile
# source /etc/profile

5.2 部署kube-controller-manager

生成kube-controller-manager证书及配置文件，配置系统服务并启动，验证组件状态：

bash 复制代码 
# 1. 生成kube-controller-manager证书
cat > /opt/cert/kube-controller-manager-csr.json << EOF
{
  "CN": "system:kube-controller-manager",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "Beijing",
      "L": "Beijing",
      "O": "system:kube-controller-manager",
      "OU": "Kubernetes"
    }
  ]
}
EOF

cfssl gencert -ca=/etc/kubernetes/pki/ca.pem \
  -ca-key=/etc/kubernetes/pki/ca-key.pem \
  -config=/opt/cert/ca-config.json \
  -profile=kubernetes /opt/cert/kube-controller-manager-csr.json | cfssljson -bare /opt/cert/kube-controller-manager
mv /opt/cert/kube-controller-manager*.pem /etc/kubernetes/pki/

# 2. kube-controller-manager.kubeconfig
cat > /etc/kubernetes/kube-controller-manager.kubeconfig << EOF
apiVersion: v1
kind: Config
clusters:
- cluster:
    certificate-authority: /etc/kubernetes/pki/ca.pem
    server: https://192.168.133.140:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: system:kube-controller-manager
  name: system:kube-controller-manager@kubernetes
current-context: system:kube-controller-manager@kubernetes
users:
- name: system:kube-controller-manager
  user:
    client-certificate: /etc/kubernetes/pki/kube-controller-manager.pem
    client-key: /etc/kubernetes/pki/kube-controller-manager-key.pem
EOF

# 3. kube-controller-manager.service
cat > /etc/systemd/system/kube-controller-manager.service << EOF
[Unit]
Description=Kubernetes Controller Manager
After=network.target kube-apiserver.service

[Service]
ExecStart=/usr/local/bin/kube-controller-manager \
  --allocate-node-cidrs=true \
  --authentication-kubeconfig=/etc/kubernetes/kube-controller-manager.kubeconfig \
  --authorization-kubeconfig=/etc/kubernetes/kube-controller-manager.kubeconfig \
  --bind-address=0.0.0.0 \
  --client-ca-file=/etc/kubernetes/pki/ca.pem \
  --cluster-cidr=10.244.0.0/16 \
  --cluster-name=cluster.local \
  --cluster-signing-cert-file=/etc/kubernetes/pki/ca.pem \
  --cluster-signing-duration=876000h \
  --cluster-signing-key-file=/etc/kubernetes/pki/ca-key.pem \
  --controllers=*,bootstrapsigner,tokencleaner \
  --feature-gates=RotateKubeletServerCertificate=true \
  --kubeconfig=/etc/kubernetes/kube-controller-manager.kubeconfig \
  --leader-elect=true \
  --node-cidr-mask-size=24 \
  --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.pem \
  --root-ca-file=/etc/kubernetes/pki/ca.pem \
  --service-account-private-key-file=/etc/kubernetes/pki/sa.key \
  --service-cluster-ip-range=10.96.0.0/16 \
  --use-service-account-credentials=true \
  --authorization-always-allow-paths=/metrics \
  --v=2
Restart=always
RestartSec=5
LimitNOFILE=65535

[Install]
WantedBy=multi-user.target
EOF

# 4. 启动 kube-controller-manager
systemctl daemon-reload && systemctl enable kube-controller-manager --now

kubectl get  cs

5.3 部署kube-scheduler

生成kube-scheduler证书及配置文件，配置系统服务并启动，验证组件状态：

bash 复制代码 
# 1. kube-scheduler 证书
cat > /opt/cert/kube-scheduler-csr.json << EOF
{
  "CN": "system:kube-scheduler",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "Beijing",
      "L": "Beijing",
      "O": "system:kube-scheduler",
      "OU": "Kubernetes"
    }
  ]
}
EOF

cfssl gencert -ca=/etc/kubernetes/pki/ca.pem \
  -ca-key=/etc/kubernetes/pki/ca-key.pem \
  -config=/opt/cert/ca-config.json \
  -profile=kubernetes /opt/cert/kube-scheduler-csr.json | cfssljson -bare /opt/cert/kube-scheduler
mv /opt/cert/kube-scheduler*.pem /etc/kubernetes/pki/

# 2. kube-scheduler.kubeconfig
cat > /etc/kubernetes/kube-scheduler.kubeconfig << EOF
apiVersion: v1
kind: Config
clusters:
- cluster:
    certificate-authority: /etc/kubernetes/pki/ca.pem
    server: https://192.168.133.140:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: system:kube-scheduler
  name: system:kube-scheduler@kubernetes
current-context: system:kube-scheduler@kubernetes
users:
- name: system:kube-scheduler
  user:
    client-certificate: /etc/kubernetes/pki/kube-scheduler.pem
    client-key: /etc/kubernetes/pki/kube-scheduler-key.pem
EOF

# 3. kube-scheduler.service
cat > /etc/systemd/system/kube-scheduler.service << EOF
[Unit]
Description=Kubernetes Scheduler
After=network.target kube-apiserver.service

[Service]
ExecStart=/usr/local/bin/kube-scheduler \
  --authentication-kubeconfig=/etc/kubernetes/kube-scheduler.kubeconfig \
  --authorization-kubeconfig=/etc/kubernetes/kube-scheduler.kubeconfig \
  --bind-address=0.0.0.0 \
  --feature-gates=RotateKubeletServerCertificate=true \
  --kubeconfig=/etc/kubernetes/kube-scheduler.kubeconfig \
  --leader-elect=true \
  --authorization-always-allow-paths=/metrics \
  --v=2
Restart=always
RestartSec=5
LimitNOFILE=65535

[Install]
WantedBy=multi-user.target
EOF

# 4. 启动 kube-scheduler
systemctl daemon-reload && systemctl enable kube-scheduler --now
systemctl status kube-scheduler

kubectl get cs
bash 复制代码 
[root@master01 ~]#
[root@master01 ~]# kubectl get cs
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME                 STATUS    MESSAGE   ERROR
etcd-0               Healthy   ok
controller-manager   Healthy   ok
scheduler            Healthy   ok
[root@master01 ~]#

六、部署kubelet（二进制方式，kubelet自动注册+生成证书+证书自动轮换）

6.1 kubelet自动注册集群（基于bootstrap token）

kubelet采用二进制部署，通过bootstrap token完成集群自动注册，结合证书自动轮换功能，无需手动干预证书更新，步骤如下：

bash 复制代码 
# 1. kubelet 工作目录和CA
mkdir -p /var/lib/kubelet/pki 
cp /etc/kubernetes/pki/ca.pem  /var/lib/kubelet/pki 

# 2. 生成kubelet配置文件（指定bootstrap token、CA证书等，实现自动注册）
kubectl create clusterrolebinding kubelet-bootstrap --clusterrole=system:node-bootstrapper --user=kubelet-bootstrap

# 3. 授权kube-controller-manager自动批准kubelet证书签名请求（CSR，实现自动签名）
kubectl create clusterrolebinding node-autoapprove-bootstrap --clusterrole=system:certificates.k8s.io:certificatesigningrequests:nodeclient --user=kubelet-bootstrap

# 4. 授权nodes用户自动轮换证书（启用kubelet证书自动轮换的核心授权）
kubectl create clusterrolebinding node-autoapprove-certificate-rotation --clusterrole=system:certificates.k8s.io:certificatesigningrequests:selfnodeclient --user=system:nodes

# 5. 生成kubelet bootstrap-kubelet.kubeconfig（用于初始注册）
cat > /etc/kubernetes/bootstrap-kubelet.kubeconfig << EOF
apiVersion: v1
kind: Config
clusters:
- cluster:
    certificate-authority: /etc/kubernetes/pki/ca.pem
    server: https://192.168.133.140:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubelet-bootstrap
  name: default
current-context: default
users:
- name: kubelet-bootstrap
  user:
    token: ed0248a55cde31f039f9c9019c3a5440  # 与token.csv中的token一致
EOF

6.2 kubelet 配置文件和启动文件

开启证书自动轮换后，kubelet会定期更新证书（默认有效期1年），无需手动操作，可通过以下命令验证轮换配置生效：

bash 复制代码 
# 1. config.yaml 配置文件
cat > /var/lib/kubelet/config.yaml << EOF
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
address: 0.0.0.0
authentication:
  anonymous:
    enabled: false
  webhook:
    cacheTTL: 2m0s
    enabled: true
  x509:
    clientCAFile: /etc/kubernetes/pki/ca.pem
authorization:
  mode: Webhook
  webhook:
    cacheAuthorizedTTL: 5m0s
    cacheUnauthorizedTTL: 30s
cgroupDriver: systemd
cgroupsPerQOS: true
clusterDNS:
- 10.96.0.2
clusterDomain: cluster.local
configMapAndSecretChangeDetectionStrategy: Watch
containerLogMaxFiles: 3
containerLogMaxSize: 10Mi
enforceNodeAllocatable:
- pods
eventBurst: 10
eventRecordQPS: 5
evictionHard:
  imagefs.available: 15%
  memory.available: 300Mi
  nodefs.available: 10%
  nodefs.inodesFree: 5%
evictionPressureTransitionPeriod: 5m0s
failSwapOn: true
fileCheckFrequency: 40s
hairpinMode: hairpin-veth
healthzBindAddress: 0.0.0.0
healthzPort: 10248
httpCheckFrequency: 40s
imageGCHighThresholdPercent: 85
imageGCLowThresholdPercent: 80
imageMinimumGCAge: 2m0s
kubeAPIBurst: 100
kubeAPIQPS: 50
makeIPTablesUtilChains: true
maxOpenFiles: 1000000
maxParallelImagePulls: 5
maxPods: 110
nodeLeaseDurationSeconds: 40
nodeStatusReportFrequency: 1m0s
nodeStatusUpdateFrequency: 10s
oomScoreAdj: -999
podPidsLimit: -1
port: 10250
# disable readOnlyPort
readOnlyPort: 10255
resolvConf: /etc/resolv.conf
runtimeRequestTimeout: 2m0s
serializeImagePulls: false
streamingConnectionIdleTimeout: 4h0m0s
syncFrequency: 1m0s
rotateCertificates: true
certificateRotationGracePeriod: 30s
EOF

# 2. 创建kubelet系统服务文件
cat > /etc/systemd/system/kubelet.service << EOF
[Unit]
Description=Kubernetes Kubelet
After=network.target containerd.service

[Service]
WorkingDirectory=/var/lib/kubelet
ExecStartPre=/bin/mount -o remount,rw '/sys/fs/cgroup'
CPUAccounting=true
MemoryAccounting=true
ExecStart=/usr/local/bin/kubelet \
  --config=/var/lib/kubelet/config.yaml \
  --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
  --hostname-override=master01 \
  --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.kubeconfig  \
  --kubeconfig=/var/lib/kubelet/kubelet.kubeconfig \
  --root-dir=/var/lib/kubelet \
  --v=2

Restart=always
StartLimitInterval=0
RestartSec=10
LimitNOFILE=65535

[Install]
WantedBy=multi-user.target
EOF

# 3. 启动kubelet服务（此时kubelet会通过bootstrap token向apiserver注册）
systemctl daemon-reload && systemctl enable kubelet --now
systemctl status kubelet

# 4. 查看节点是否已自动注册到集群(看到节点表示自动注册成功)
kubectl get no

七、部署kube-proxy（两种部署方式，二选一）

kube-proxy提供两种部署方式：二进制部署（适合单机场景，便于手动管理）、DaemonSet部署（适合集群扩展，自动调度），两种方式相互独立，选择一种即可。

7.1 方式一：二进制部署kube-proxy

生成kube-proxy证书及配置文件，配置系统服务，启动后验证服务状态：

bash 复制代码 
# 1. 生成kube-proxy证书
cat > /opt/cert/kube-proxy-csr.json << EOF
{
  "CN": "system:kube-proxy",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "Beijing",
      "L": "Beijing",
      "O": "system:node-proxier",
      "OU": "Kubernetes"
    }
  ]
}
EOF

cfssl gencert -ca=/etc/kubernetes/pki/ca.pem \
  -ca-key=/etc/kubernetes/pki/ca-key.pem \
  -config=/opt/cert/ca-config.json \
  -profile=kubernetes /opt/cert/kube-proxy-csr.json | cfssljson -bare /opt/cert/kube-proxy
mv /opt/cert/kube-proxy*.pem /etc/kubernetes/pki/

# 2. 生成kube-proxy kubeconfig配置文件
cat > /etc/kubernetes/kube-proxy.kubeconfig << EOF
apiVersion: v1
kind: Config
clusters:
- cluster:
    certificate-authority: /etc/kubernetes/pki/ca.pem
    server: https://192.168.133.140:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: system:kube-proxy
  name: system:kube-proxy@kubernetes
current-context: system:kube-proxy@kubernetes
users:
- name: system:kube-proxy
  user:
    client-certificate: /etc/kubernetes/pki/kube-proxy.pem
    client-key: /etc/kubernetes/pki/kube-proxy-key.pem
EOF

# 3. 生成kube-proxy配置文件（指定ipvs模式，适配前文ipvs模块）
mkdir -p /var/lib/kube-proxy

cat > /var/lib/kube-proxy/kube-proxy-config.yaml << EOF
kind: KubeProxyConfiguration
apiVersion: kubeproxy.config.k8s.io/v1alpha1
bindAddress: 0.0.0.0
clientConnection:
  kubeconfig: "/etc/kubernetes/kube-proxy.kubeconfig"
clusterCIDR: "10.244.0.0/16"
conntrack:
  maxPerCore: 32768
  min: 131072
  tcpCloseWaitTimeout: 1h0m0s
  tcpEstablishedTimeout: 24h0m0s
healthzBindAddress: 0.0.0.0:10256
hostnameOverride: "master01"  # 主机名
metricsBindAddress: 0.0.0.0:10249
mode: "ipvs"
ipvs:
  scheduler: "rr"
EOF

# 4. 创建kube-proxy系统服务文件
cat > /etc/systemd/system/kube-proxy.service << EOF
[Unit]
Description=Kubernetes Kube-Proxy
After=network.target kube-apiserver.service

[Service]
ExecStart=/usr/local/bin/kube-proxy \
  --config=/etc/kubernetes/kube-proxy-config.yaml \
  --v=2
Restart=always
RestartSec=5
LimitNOFILE=65535

[Install]
WantedBy=multi-user.target
EOF

# 5. 启动kube-proxy并验证
systemctl daemon-reload && systemctl enable kube-proxy --now
systemctl status kube-proxy

# 验证ipvs规则（可看到kube-proxy生成的ipvs规则）
ipvsadm -Ln

7.2 方式二：DaemonSet部署kube-proxy（推荐集群扩展）

通过DaemonSet方式部署kube-proxy，会自动在所有节点（此处仅master节点）部署，无需手动配置系统服务，步骤如下：

bash 复制代码 
# 1. 生成kube-proxy配置文件（通过ConfigMap挂载）
cat > kube-proxy-configmap.yaml << EOF
apiVersion: v1
data:
  config.conf: |-
    apiVersion: kubeproxy.config.k8s.io/v1alpha1
    bindAddress: 0.0.0.0
    clientConnection:
      kubeconfig: /var/lib/kube-proxy/kubeconfig.conf
    clusterCIDR: 10.244.0.0/16
    healthzBindAddress: ""
    hostnameOverride: ""
    ipvs:
      scheduler: "rr"
    kind: KubeProxyConfiguration
    logging:
      flushFrequency: 0
      options:
        json:
          infoBufferSize: "0"
      verbosity: 0
    metricsBindAddress: ""
    mode: ipvs
    oomScoreAdj: -999
  kubeconfig.conf: |-
    apiVersion: v1
    kind: Config
    clusters:
    - cluster:
        certificate-authority: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
        server: https://192.168.133.140:6443
      name: default
    contexts:
    - context:
        cluster: default
        namespace: default
        user: default
      name: default
    current-context: default
    users:
    - name: default
      user:
        tokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token
kind: ConfigMap
metadata:
  labels:
    app: kube-proxy
  name: kube-proxy
  namespace: kube-system
EOF

# 2. 创建DaemonSet部署kube-proxy
cat > kube-proxy-daemonset.yaml << EOF
---
# kube-proxy ServiceAccount 配置
apiVersion: v1
kind: ServiceAccount
metadata:
  name: kube-proxy
  namespace: kube-system
  labels:
    k8s-app: kube-proxy
---
# kube-proxy 集群角色绑定配置
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kube-proxy
  labels:
    k8s-app: kube-proxy
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:node-proxier  # kube-proxy 必需的内置集群角色
subjects:
- kind: ServiceAccount
  name: kube-proxy
  namespace: kube-system  # 与上面的 ServiceAccount 命名空间一致
---

---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  labels:
    k8s-app: kube-proxy
  name: kube-proxy
  namespace: kube-system
spec:
  selector:
    matchLabels:
      k8s-app: kube-proxy
  template:
    metadata:
      labels:
        k8s-app: kube-proxy
    spec:
      containers:
      - command:
        - /usr/local/bin/kube-proxy
        - --config=/var/lib/kube-proxy/config.conf
        - --hostname-override=\$(NODE_NAME)
        env:
        - name: TZ  
          value: Asia/Shanghai
        - name: NODE_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: spec.nodeName
        #image: registry.aliyuncs.com/google_containers/kube-proxy:v1.28.7
        image: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.28.7
        imagePullPolicy: IfNotPresent
        name: kube-proxy
        resources: {}
        securityContext:
          privileged: true
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /var/lib/kube-proxy
          name: kube-proxy
        - mountPath: /run/xtables.lock
          name: xtables-lock
        - mountPath: /lib/modules
          name: lib-modules
          readOnly: true
      dnsPolicy: ClusterFirst
      hostNetwork: true
      nodeSelector:
        kubernetes.io/os: linux
      priorityClassName: system-node-critical
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: kube-proxy
      serviceAccountName: kube-proxy
      tolerations:
      - operator: Exists
      volumes:
      - configMap:
          defaultMode: 420
          name: kube-proxy
        name: kube-proxy
      - hostPath:
          path: /run/xtables.lock
          type: FileOrCreate
        name: xtables-lock
      - hostPath:
          path: /lib/modules
          type: ""
        name: lib-modules
---
EOF

# 3. 执行部署命令
kubectl apply -f kube-proxy-configmap.yaml
kubectl apply -f kube-proxy-daemonset.yaml

# 4. 验证部署结果（确保Pod处于Running状态）
kubectl get pods -n kube-system | grep kube-proxy

# 验证ipvs规则
ipvsadm -Ln

八、部署CNI网络插件（必部署）

CNI网络插件用于实现Pod之间的网络通信，此处选择calico，部署步骤如下：

bash 复制代码 
# 1. 下载calico部署文件（v3.31.4版本）
wget https://raw.githubusercontent.com/projectcalico/calico/v3.31.4/manifests/calico.yaml

# 2. 修改calico配置，指定集群CIDR（与kube-controller-manager中的cluster-cidr一致）
sed -i 's/192.168.0.0\/16/10.244.0.0\/16/g' calico.yaml
sed -i 's/docker.io/mirror.ccs.tencentyun.com/g' calico.yaml

# calico 用到的镜像可替换成以下的镜像
# docker pull registry.cn-shenzhen.aliyuncs.com/rmyyds/cni:v3.31.4
# docker pull registry.cn-shenzhen.aliyuncs.com/rmyyds/node:v3.31.4
# docker pull registry.cn-shenzhen.aliyuncs.com/rmyyds/kube-controllers:v3.31.4

# 3. 部署calico
kubectl apply -f calico.yaml

# 4. 验证calico部署（所有calico相关Pod均为Running状态即为正常）
kubectl get pods -n kube-system

# 5. 部署完成后，查看Node状态（变为Ready即为正常）
kubectl get nodes

九、部署coredns插件（必部署）

coredns用于k8s集群内部服务解析,部署步骤如下：

bash 复制代码 
# 1. 下载coredns部署文件
curl -O https://raw.githubusercontent.com/coredns/deployment/master/kubernetes/coredns.yaml.sed
curl -O https://raw.githubusercontent.com/coredns/deployment/master/kubernetes/deploy.sh

# 2. 修改coredns配置，生成部署文件（--dns-server 指定CoreDNS的ClusterIP为10.96.0.2）
chmod +x deploy.sh
./deploy.sh -i 10.96.0.2 > coredns.yaml

# 3. 替换成阿里云镜像
sed -i 's/coredns\/coredns/registry.aliyuncs.com\/google_containers\/coredns/g' coredns.yaml
# docker pull registry.aliyuncs.com/google_containers/coredns:1.9.4


# 4. 部署coredns（所有相关Pod均为Running状态即为正常）
kubectl apply -f coredns.yaml

# 5. 验证coredns部署（所有coredns相关Pod均为Running状态即为正常）
kubectl get pod -A

十、集群部署完成验证

所有组件部署完成后，执行以下命令验证集群整体状态，确保所有组件正常运行：

bash 复制代码 
# 1. 查看集群组件状态（etcd、controller-manager、scheduler均为Healthy）
kubectl get cs

# 2. 查看节点状态（node状态为Ready）
kubectl get nodes -o wide

# 3. 查看所有命名空间下的Pod状态（所有Pod均为Running，无Error、CrashLoopBackOff）
kubectl get pods --all-namespaces

# 4. 查看集群信息
kubectl cluster-info

# 5. 验证核心组件日志（可选，若有异常可查看日志排查）
systemctl status etcd containerd kube-apiserver kube-controller-manager kube-scheduler kubelet kube-proxy -l

# 6. 验证Pod网络通信（再次测试，确保跨Pod通信正常）
kubectl run test-pod1 --image=busybox:1.35 --command -- sleep 3600
kubectl run test-pod2 --image=busybox:1.35 --command -- sleep 3600
# 获取test-pod2的IP
POD2_IP=$(kubectl get pod test-pod2 -o jsonpath='{.status.podIP}')
# 在test-pod1中ping test-pod2的IP
kubectl exec -it test-pod1 -- ping $POD2_IP -c 3
# 删除测试Pod
kubectl delete pod test-pod1 test-pod2

十一、常见问题排查

11.1 组件启动失败

通过以下命令查看组件日志，定位失败原因：

bash 复制代码 
# 查看指定组件日志（以kubelet为例）
journalctl -u kubelet -f
# 查看容器日志（以calico为例）
kubectl logs -n kube-system <calico-pod-name> -f

11.2 kubelet注册失败

常见原因：bootstrap token不一致、CSR未批准、网络不通，排查步骤：

bash 复制代码 
# 1. 确认bootstrap token与token.csv一致
cat /etc/kubernetes/token.csv | awk -F ',' '{print $1}'
cat /etc/kubernetes/kubelet/bootstrap-kubeconfig | grep token

# 2. 确认CSR已批准
kubectl get csr

# 3. 确认kubelet与apiserver网络连通（6443端口可通）
telnet 192.168.133.140 6443

11.3 Pod网络不通

常见原因：calico未正常启动、ipvs模块未加载，排查步骤：

bash 复制代码 
# 1. 确认calico所有Pod正常运行
kubectl get pods -n kube-system

# 2. 确认ipvs模块已加载
lsmod | grep ip_vs

# 3. 查看calico节点状态
kubectl exec -n kube-system <calico-node-pod-name> -- calicoctl node status

11.4 单机部署k8s，要去除master节点的不可调度策略和去除污点

bash 复制代码 
kubectl taint nodes master01 node.kubernetes.io/not-ready-
kubectl taint nodes master01 node-role.kubernetes.io/control-plane:NoSchedule-
kubectl taint nodes master01 node-role.kubernetes.io/master:NoSchedule-

十二、总结

本文档完成了K8s v1.28.7 单Master节点纯二进制部署，涵盖环境准备、证书生成、核心组件部署、kubelet自动注册、kube-proxy两种部署方式及CNI网络插件部署，所有命令可直接复制执行。部署完成后，集群可正常运行，支持Pod创建、网络通信等核心功能，适合单机测试、小型生产环境使用。若需扩展为多节点集群，可参考本文档步骤，在新增节点上部署kubelet、kube-proxy及calico即可。

相关推荐
wxjlkh12 小时前
5分钟部署Docker！Rocky Linux极速安装+一键加速配置脚本
云原生·eureka
丈剑走天涯12 小时前
kubernetes java app 部署使用harbor私服 问题集合
java·容器·kubernetes
Jinkxs14 小时前
Java 部署：滚动更新（K8s RollingUpdate 策略）
java·开发语言·kubernetes
lpfasd12315 小时前
Kubernetes (K8s) 底层早已不再直接使用 Docker 引擎了
java·docker·kubernetes
不吃香菜kkk、16 小时前
通过夜莺n9e监控Kubernetes集群
安全·云原生·容器·kubernetes
淡泊if17 小时前
K8s 网络排障：从抓包开始，一步步定位诡异“502”
网络·kubernetes·k8s
小李的便利店17 小时前
k8s集群部署Prometheus和Grafana
kubernetes·grafana·prometheus·监控
margu_16818 小时前
【Docker】nscenter命令详解
运维·docker·容器
阿里云云原生18 小时前
Higress 加入 CNCF：保障 Nginx Ingress 迁移，提供企业级 AI 网关
云原生
热门推荐
01GitHub 镜像站点02Qwen3.5 开源全解析：从 0.8B 到 397B，代际升级 + 全场景选型指南03OpenClaw 使用和管理 MCP 完全指南04UV安装并设置国内源05OpenClaw Control UI安全上下文访问配置06小黑课堂计算机二级WPSoffice题库软件下载安装教程（2026年3月最新版）07Claude Code + GLM4.7 避坑指南：解决 Unable to connect to Anthropic services08OpenClaw macOS 完整安装与本地模型配置教程（实战版）09本地部署 OpenClaw + DeepSeek-R1 完全指南10AI 编程三剑客：Spec-Kit、OpenSpec、Superpowers 深度对比与实战指南