k8s-动态卷供应

Kubernetes 动态卷供应

动态卷介绍

静态 PV/PVC(传统手动方式):必须管理员提前手动写 YAML 创建 PV,用户再创建 PVC 去匹配 PV;没有 PV 就会一直 Pending。

动态供给(StorageClass + PVC)不用手动创建 PV,只需要创建 PVC,集群自动生成 PV 并绑定,也就是动态分配 PV。

这就是动态卷供应的能力。它彻底省去了集群管理员提前手动配置存储的工作,用户只要提出存储需求,系统就会自动创建对应的存储卷。

动态卷供应流程

  1. 管理员先创建 "存储模板"(StorageClass),指定用哪个 "造卷工具"(Provisioner,制备器)来创建卷;
  2. 用户创建 "存储申请单"(PVC)时,指定要用哪个 "存储模板";
  3. 系统收到申请后,会让模板绑定的 "造卷工具" 自动创建一个 PV,并且把 PV 和用户的 PVC 绑定,用户直接用 PVC 就行。

StorageClass

StorageClass 就像管理员定义的 "存储套餐"------ 不同套餐对应不同的存储类型、服务质量(比如读写速度)、备份策略等。Kubernetes 不关心套餐叫什么,只负责按套餐规则造卷。

  • 命名很重要:用户创建 PVC 时,要通过这个名字选择对应的存储套餐;
  • 创建后不能改:StorageClass 一旦创建,名字和参数就没法修改了,要改只能重新建。

核心配置

每个 "存储套餐" 都包含以下核心信息,用来指导系统创建 PV:

  • provisioner:造卷工具(制备器),指定用哪个工具创建 PV,必须设置。

    Kubernetes 自带一些 Provisioner,名字以 "kubernetes.io" 开头(比如 AWS EBS、Azure Disk);也可以使用第三方提供的(比如 NFS 没有内置工人,需要自己装)Provisioner,按 Kubernetes 规则运行即可。

    存储类型 是否内置 说明
    AWSElasticBlockStore ✅ 是 亚马逊云硬盘
    AzureFile/AzureDisk ✅ 是 微软云文件 / 硬盘
    Ceph RBD ✅ 是 Ceph 块存储
    Glusterfs ✅ 是 分布式文件系统
    NFS ❌ 否 网络文件系统,需装外部工人
    本地存储(Local) ❌ 否 节点本地硬盘,需装外部工人
    iSCSI/CephFS ❌ 否 需外部工人
  • reclaimPolicy:PV 回收策略,指定 PV 不用怎么处理 PV。

    • Delete(默认):PVC 删除后,PV 也自动删,对应的后端存储(比如云硬盘、NFS 目录)也会清掉;
    • Retain:PVC 删除后,PV 保留下来,后端存储的数据也不删,管理员可以手动处理。
  • volumeBindingMode:PVC 创建出来后,指定 PVC 与P V 绑定时机。

    • Immediate(立即绑定):创建 PVC 就造卷,不管 Pod 会不会用;适合存储能被所有节点访问的场景(比如云硬盘);
    • WaitForFirstConsumer(等使用者):先不造卷,等第一个用这个 PVC 的 Pod 创建后,根据 Pod 要跑的节点造卷;适合本地存储、只能被特定节点访问的存储。
  • mountOptions:指定 PV 挂载时附加的 "额外设置",比如权限、调试模式。

    注意:如果存储不支持这些选项,造卷会失败;如果选项填错,PV 挂载会失败。

  • parameters:比如存储类型(SSD/HDD)、副本数等,不同工具参数不一样。

基础示例

yaml 复制代码
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: standard  # 套餐名,用户PVC要填这个
provisioner: kubernetes.io/aws-ebs  # 用AWS EBS的造卷工具
parameters:
  type: gp2  # 存储类型为gp2(AWS的通用型SSD)
reclaimPolicy: Retain  # PVC 不用了,pv保留
allowVolumeExpansion: true  # 允许扩容
mountOptions:
  - debug  # 挂载时开启调试模式
volumeBindingMode: Immediate  # 立即创建并绑定PV

部署 Local Path provisioner

Local Path Provisioner 是 K8s 官方推荐的本地存储动态卷插件,适合需要使用节点本地磁盘的场景。

部署本地路径制备器

bash 复制代码
# 下载官方模板
root@master30 ~ 13:25:02# wget https://raw.githubusercontent.com/rancher/local-path-provisioner/master/deploy/local-path-storage.yaml
#可以以直接在浏览器搜
# 查看资源配置
root@master30 ~ 13:25:35# vim local-path-storage.yaml
yaml 复制代码
apiVersion: v1
kind: Namespace
metadata:
  name: local-path-storage

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: local-path-provisioner-service-account
  namespace: local-path-storage

---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: local-path-provisioner-role
  namespace: local-path-storage
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list", "watch", "create", "patch", "update", "delete"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: local-path-provisioner-role
rules:
  - apiGroups: [""]
    resources: ["nodes", "persistentvolumeclaims", "configmaps", "pods", "pods/log"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "patch", "update", "delete"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "patch"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: local-path-provisioner-bind
  namespace: local-path-storage
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: local-path-provisioner-role
subjects:
  - kind: ServiceAccount
    name: local-path-provisioner-service-account
    namespace: local-path-storage

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: local-path-provisioner-bind
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: local-path-provisioner-role
subjects:
  - kind: ServiceAccount
    name: local-path-provisioner-service-account
    namespace: local-path-storage

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: local-path-provisioner
  namespace: local-path-storage
spec:
  replicas: 1
  selector:
    matchLabels:
      app: local-path-provisioner
  template:
    metadata:
      labels:
        app: local-path-provisioner
    spec:
      serviceAccountName: local-path-provisioner-service-account
      containers:
        - name: local-path-provisioner
          image: rancher/local-path-provisioner:v0.0.35
          imagePullPolicy: IfNotPresent
          command:
            - local-path-provisioner
            - --debug
            - start
            - --config
            - /etc/config/config.json
          volumeMounts:
            - name: config-volume
              mountPath: /etc/config/
          env:
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: CONFIG_MOUNT_PATH
              value: /etc/config/
      volumes:
        - name: config-volume
          configMap:
            name: local-path-config

---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: local-path
provisioner: rancher.io/local-path
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Delete

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: local-path-config
  namespace: local-path-storage
data:
  config.json: |-
    {
            "nodePathMap":[
            {
                    "node":"DEFAULT_PATH_FOR_NON_LISTED_NODES",
                    "paths":["/opt/local-path-provisioner"]
            }
            ]
    }
  setup: |-
    #!/bin/sh
    set -eu
    mkdir -m 0777 -p "$VOL_DIR"
  teardown: |-
    #!/bin/sh
    set -eu
    rm -rf "$VOL_DIR"
  helperPod.yaml: |-
    apiVersion: v1
    kind: Pod
    metadata:
      name: helper-pod
    spec:
      priorityClassName: system-node-critical
      tolerations:
        - key: node.kubernetes.io/disk-pressure
          operator: Exists
          effect: NoSchedule
      containers:
      - name: helper-pod
        image: busybox
        imagePullPolicy: IfNotPresent

local-path-storage.yaml 核心由 5 类资源 组成,每部分各司其职:

资源类型 名称 核心作用
Namespace local-path-storage 隔离 Local Path Provisioner 相关资源
ConfigMap local-path-config 定义本地存储路径 规则(如默认 /opt/local-path-provisioner
ServiceAccount + RBAC local-path-provisioner 赋予 Provisioner 操作 PV/PVC 的权限
Deployment local-path-provisioner 运行 Local Path Provisioner 核心程序(动态创建本地 PV)
StorageClass local-path 提供给 PVC 调用的「本地存储模板」
bash 复制代码
# 资源默认部署在命名空间:local-path-storage
root@master30 ~ 13:26:19# kubectl apply -f local-path-storage.yaml

# 确认资源状态
root@master30 ~ 13:27:16# kubectl get sc local-path 
NAME         PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
local-path   rancher.io/local-path   Delete          WaitForFirstConsumer   false                  67m

root@master30 ~ 13:27:02# kubectl get all -n local-path-storage 
NAME                                          READY   STATUS    RESTARTS   AGE
pod/local-path-provisioner-85d9db4886-5t4sn   1/1     Running   0          14s

NAME                                     READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/local-path-provisioner   1/1     1            1           14s

NAME                                                DESIRED   CURRENT   READY   AGE
replicaset.apps/local-path-provisioner-85d9db4886   1         1         1       14s

验证部署

bash 复制代码
# 设置默认命名空间
root@master30 ~ 13:34:45# kubectl config set-context --current --namespace default
1. 创建 pvc
bash 复制代码
#向名为local‑path的存储类申请5Gi本地存储,访问模式为单节点读写,放在 default 命名空间,由 local‑path provisioner 自动生成绑定 PV,供 Pod 挂载持久化数据。
root@master30 ~ 13:35:01# cat > local-path-pvc.yaml <<'EOF'
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: local-path-pvc
  namespace: default
spec:
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 5Gi
  storageClassName: local-path  # 指定 Local Path 的 StorageClass
EOF


root@master30 ~ 13:36:28# kubectl apply -f local-path-pvc.yaml
root@master30 ~ 13:37:49# kubectl get pvc
NAME             STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
local-path-pvc   Pending                                      local-path     <unset>                 14s

# 等待pod关联才会创建本地目录
2. 创建 deployment
bash 复制代码
root@master30 ~ 13:38:03#  cat > local-path-deployment.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: webapp
  name: webapp
  namespace: default
spec:
  replicas: 2
  selector:
    matchLabels:
      app: webapp
  template:
    metadata:
      labels:
        app: webapp
    spec:
      containers:
      - image: nginx
        name: nginx
        volumeMounts:
        - name: webapp-data
          mountPath: /usr/share/nginx/html
      volumes:
      - name: webapp-data
        persistentVolumeClaim:
          claimName: local-path-pvc
EOF

root@master30 ~ 13:38:18#  kubectl apply -f local-path-deployment.yaml

# 查看 pod 状态
root@master30 ~ 13:38:56# kubectl get pods -o wide 
NAME                      READY   STATUS    RESTARTS   AGE   IP              NODE                 NOMINATED NODE   READINESS GATES
webapp-6d46b54487-92cpv   1/1     Running   0          30s   10.224.26.130   worker31.ggg.cloud   <none>           <none>
webapp-6d46b54487-frcvf   1/1     Running   0          30s   10.224.26.191   worker31.ggg.cloud   <none>           <none>
# 所有节点都调度到 worker31,因为只用worker31可以提供这个卷。

# 查看 pvc 状态
root@master30 ~ 13:38:58# kubectl get pvc
NAME             STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
local-path-pvc   Bound    pvc-756f682b-4e9c-4307-a1af-0e315cb282a7   5Gi        RWO            local-path     <unset>                 88s

# 写入测试文件
root@master30 ~ 13:40:24# kubectl exec webapp-6d46b54487-92cpv -- bash -c 'echo Hello World From Nginx > /usr/share/nginx/html/index.html'

#查看本地路径
root@master30 ~ 13:41:24# vim local-path-storage.yaml 
....
kind: ConfigMap
....

                    "paths":["/opt/local-path-provisioner"]
.....

# 验证本地路径
root@worker31 ~ 12:12:53# ls /opt/local-path-provisioner/pvc-756f682b-4e9c-4307-a1af-0e315cb282a7_default_local-path-pvc/
index.html

root@worker31 ~ 13:43:46# cat /opt/local-path-provisioner/pvc-756f682b-4e9c-4307-a1af-0e315cb282a7_default_local-path-pvc/index.html 
Hello World From Nginx
3. 清理资源
bash 复制代码
# 删除 deployment
root@master30 ~ 13:43:28# kubectl delete deployments.apps webapp 

# 删除 pvc
root@master30 ~ 13:44:20# kubectl delete pvc local-path-pvc 

# 本地目录也被删除
root@worker31 ~ 13:44:00# ls /opt/local-path-provisioner/pvc-756f682b-4e9c-4307-a1af-0e315cb282a7_default_local-path-pvc/
ls: cannot access '/opt/local-path-provisioner/pvc-756f682b-4e9c-4307-a1af-0e315cb282a7_default_local-path-pvc/': No such file or directory

部署 NFS provisioner

部署 NFS 服务

bash 复制代码
# 安装 NFS server
root@master30 ~ 13:45:35# apt install -y nfs-kernel-server

# 安创建NFS目录 修改创建文件夹的权限
root@master30 ~ 13:45:35# mkdir -m 777 /shares/

# 配置共享,允许所有客户端访问
root@master30 ~ 13:50:25# cat << EOF > /etc/exports
/shares *(rw,sync,no_root_squash,no_all_squash,insecure)
EOF

# 重启 nfs server
root@master30 ~ 13:50:37# systemctl restart nfs-server.service

# 客户端安装
root@worker31 ~ 13:51:03# apt install -y nfs-common
root@worker32 ~ 13:51:07# apt install -y nfs-common

部署 NFS provisioner

NFS没有内置制备器,这里我们自定义外部分配器。

bash 复制代码
# 资源部署到命名空间:kube-storage
root@master30 ~ 13:50:50# kubectl create ns kube-storage
root@master30 ~ 13:51:33#  kubectl config set-context --current --namespace kube-storage
1. 创建 RBAC 权限

创建 nfs-rbac.yaml 文件,赋予 Provisioner 操作 PV/PVC 的权限。

yaml 复制代码
root@master30:~# cat > nfs-rbac.yaml <<'EOF'
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  namespace: kube-storage
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    namespace: kube-storage
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
EOF
bash 复制代码
root@master30 ~ 13:51:56# kubectl apply -f nfs-rbac.yaml
2. 部署 NFS Provisioner 应用
yaml 复制代码
root@master30:~# cat > nfs-provisioner.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  namespace: kube-storage
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      volumes:
      - name: nfs-client-root
        nfs:
          server: 10.1.8.30  # 同上 NFS 服务器 IP
          path: /shares  # 核心:同步改为 /shares
      containers:
      - name: nfs-client-provisioner
        image: registry.k8s.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2
        volumeMounts:
        - name: nfs-client-root
          mountPath: /persistentvolumes
        env:
        - name: PROVISIONER_NAME
          value: fuseim.pri/ifs
        - name: NFS_SERVER
          value: 10.1.8.30  # 替换为你的 NFS 服务器 IP
        - name: NFS_PATH
          value: /shares  # 核心:共享路径改为 /shares
EOF
bash 复制代码
#  部署 NFS Provisioner 应用
root@master30 ~ 13:52:48# kubectl apply -f nfs-provisioner.yaml

# 查看 NFS Provisioner 应用
root@master30 ~ 13:53:49# kubectl get deployments.apps -n kube-storage
NAME                     READY   UP-TO-DATE   AVAILABLE   AGE
nfs-client-provisioner   1/1     1            1           50s
3. 创建 NFS StorageClass
yaml 复制代码
root@master30:~# cat > nfs-storageclass.yaml <<'EOF'
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-storage
  namespace: kube-storage
provisioner: fuseim.pri/ifs  # 必须和 Provisioner 名称一致
parameters:
  onDelete: retain  # 删除 PVC 保留 NFS 数据
  archiveOnDelete: "false"
# 按需设置回收策略
#reclaimPolicy: Retain
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: Immediate
EOF
bash 复制代码
# 创建 StorageClass
root@master30 ~ 13:54:07# kubectl apply -f nfs-storageclass.yaml

# 查看 StorageClass
root@master30 ~ 13:54:20# kubectl get sc
NAME          PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
nfs-storage   fuseim.pri/ifs          Delete          Immediate              true                   7s

验证部署

在default命名空间测试。

bash 复制代码
root@master30 ~ 13:54:56# kubectl config set-context --current --namespace default
1. 创建 pvc
bash 复制代码
root@master30 ~ 13:55:19#  cat > nfs-pvc.yaml <<'EOF'
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: webclaim
  namespace: default
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  storageClassName: "nfs-storage"
EOF

# 创建 pvc
root@master30 ~ 13:55:32# kubectl apply -f nfs-pvc.yaml 

# 查看 pvc 状态
root@master30 ~ 13:55:40# kubectl get pvc
NAME       STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
webclaim   Bound    pvc-7c9258e7-7589-4ecf-bce1-85611518a351   5Gi        RWO            nfs-storage    <unset>                 12s

# 查看 pv 状态
root@master30 ~ 13:55:52# kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM              STORAGECLASS   VOLUMEATTRIBUTESCLASS   REASON   AGE
pvc-7c9258e7-7589-4ecf-bce1-85611518a351   5Gi        RWO            Delete           Bound    default/webclaim   nfs-storage    <unset>                          24s

# 查看 nfs 共享目录
root@master30 ~ 13:56:04# ls /shares/
default-webclaim-pvc-7c9258e7-7589-4ecf-bce1-85611518a351

# 准备 web 主页
root@master30 ~ 13:56:33# echo Hello World From Nginx > /shares/default-webclaim-pvc-7c9258e7-7589-4ecf-bce1-85611518a351/index.html
2. 创建 deployment
bash 复制代码
root@master30 ~ 13:56:42# cat > nfs-deploy.yaml << EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: webapp
  name: webapp
spec:
  replicas: 2
  selector:
    matchLabels:
      app: webapp
  template:
    metadata:
      labels:
        app: webapp
    spec:
      containers:
      - image: nginx
        name: nginx
        volumeMounts:
        - name: webapp-data
          mountPath: /usr/share/nginx/html
      volumes:
      - name: webapp-data
        persistentVolumeClaim:
          claimName: webclaim
EOF

root@master30 ~ 13:56:52#  kubectl apply -f nfs-deploy.yaml

root@master30 ~ 13:57:00#  kubectl expose deployment webapp --target-port 80 --port 80 --protocol TCP
root@master30 ~ 13:57:05#  kubectl get svc webapp 
NAME     TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
webapp   ClusterIP   10.99.50.211   <none>        80/TCP    13s

# 访问测试
root@master30 ~ 13:57:18# curl 10.99.50.211
Hello World From Nginx
root@master30 ~ 13:57:35# curl 10.99.50.211
Hello World From Nginx
3. 清理资源
bash 复制代码
# 删除 deployment
root@master30 ~ 13:57:36# kubectl delete deployments.apps webapp

# 删除 pvc,pv也将一起删除
root@master30 ~ 13:57:48# kubectl delete pvc webclaim 

root@master30 ~ 13:57:59# kubectl get pv
No resources found


# 查看后端存储中数据,仍然保留
root@master30 ~ 13:58:23# ls /shares/default-webclaim-pvc-7c9258e7-7589-4ecf-bce1-85611518a351/
index.html

保留 NFS provisioner ,下一章 statefulset 使用。

相关推荐
yunwei371 小时前
eBPF 教程:使用 fsession 追踪慢速 vfs_read 调用
linux·云原生
叶 落2 小时前
Docker-Compose 安装 Milvus
docker·容器·milvus
深念Y3 小时前
Windows → WSL2 全面迁移:工具链、Docker、OpenCode
linux·运维·windows·docker·容器·环境·wsl
heimeiyingwang14 小时前
【架构实战】Kubernetes存储实战:从EmptyDir到Ceph CSI的持久化存储选型指南
ceph·架构·kubernetes
QX_hao15 小时前
【Kubernetes】 的三个Probe
容器·kubernetes
Henry-SAP18 小时前
SAP S/4HANA引领物流ERP新生态
人工智能·云原生·sap·erp
这个DBA有点耶18 小时前
从DBA到数据架构师(五):数据架构演进中的技术债务管理
数据库·程序人生·云原生·架构·dba·数据库管理员
猫吃了源码19 小时前
k9s简介和安装
linux·docker·kubernetes
云原生指北20 小时前
Docker Sandboxes 上手:从安装到第一次跑 Agent
运维·docker·容器