Kubernetes 动态卷供应
动态卷介绍
静态 PV/PVC(传统手动方式):必须管理员提前手动写 YAML 创建 PV,用户再创建 PVC 去匹配 PV;没有 PV 就会一直 Pending。
动态供给(StorageClass + PVC) :不用手动创建 PV,只需要创建 PVC,集群自动生成 PV 并绑定,也就是动态分配 PV。
这就是动态卷供应的能力。它彻底省去了集群管理员提前手动配置存储的工作,用户只要提出存储需求,系统就会自动创建对应的存储卷。
动态卷供应流程
- 管理员先创建 "存储模板"(StorageClass),指定用哪个 "造卷工具"(Provisioner,制备器)来创建卷;
- 用户创建 "存储申请单"(PVC)时,指定要用哪个 "存储模板";
- 系统收到申请后,会让模板绑定的 "造卷工具" 自动创建一个 PV,并且把 PV 和用户的 PVC 绑定,用户直接用 PVC 就行。
StorageClass
StorageClass 就像管理员定义的 "存储套餐"------ 不同套餐对应不同的存储类型、服务质量(比如读写速度)、备份策略等。Kubernetes 不关心套餐叫什么,只负责按套餐规则造卷。
- 命名很重要:用户创建 PVC 时,要通过这个名字选择对应的存储套餐;
- 创建后不能改:StorageClass 一旦创建,名字和参数就没法修改了,要改只能重新建。
核心配置
每个 "存储套餐" 都包含以下核心信息,用来指导系统创建 PV:
-
provisioner:造卷工具(制备器),指定用哪个工具创建 PV,必须设置。
Kubernetes 自带一些 Provisioner,名字以 "kubernetes.io" 开头(比如 AWS EBS、Azure Disk);也可以使用第三方提供的(比如 NFS 没有内置工人,需要自己装)Provisioner,按 Kubernetes 规则运行即可。
存储类型 是否内置 说明 AWSElasticBlockStore ✅ 是 亚马逊云硬盘 AzureFile/AzureDisk ✅ 是 微软云文件 / 硬盘 Ceph RBD ✅ 是 Ceph 块存储 Glusterfs ✅ 是 分布式文件系统 NFS ❌ 否 网络文件系统,需装外部工人 本地存储(Local) ❌ 否 节点本地硬盘,需装外部工人 iSCSI/CephFS ❌ 否 需外部工人 -
reclaimPolicy:PV 回收策略,指定 PV 不用怎么处理 PV。
- Delete(默认):PVC 删除后,PV 也自动删,对应的后端存储(比如云硬盘、NFS 目录)也会清掉;
- Retain:PVC 删除后,PV 保留下来,后端存储的数据也不删,管理员可以手动处理。
-
volumeBindingMode:PVC 创建出来后,指定 PVC 与P V 绑定时机。
- Immediate(立即绑定):创建 PVC 就造卷,不管 Pod 会不会用;适合存储能被所有节点访问的场景(比如云硬盘);
- WaitForFirstConsumer(等使用者):先不造卷,等第一个用这个 PVC 的 Pod 创建后,根据 Pod 要跑的节点造卷;适合本地存储、只能被特定节点访问的存储。
-
mountOptions:指定 PV 挂载时附加的 "额外设置",比如权限、调试模式。
注意:如果存储不支持这些选项,造卷会失败;如果选项填错,PV 挂载会失败。
-
parameters:比如存储类型(SSD/HDD)、副本数等,不同工具参数不一样。
基础示例
yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: standard # 套餐名,用户PVC要填这个
provisioner: kubernetes.io/aws-ebs # 用AWS EBS的造卷工具
parameters:
type: gp2 # 存储类型为gp2(AWS的通用型SSD)
reclaimPolicy: Retain # PVC 不用了,pv保留
allowVolumeExpansion: true # 允许扩容
mountOptions:
- debug # 挂载时开启调试模式
volumeBindingMode: Immediate # 立即创建并绑定PV
部署 Local Path provisioner
Local Path Provisioner 是 K8s 官方推荐的本地存储动态卷插件,适合需要使用节点本地磁盘的场景。
部署本地路径制备器
bash
# 下载官方模板
root@master30 ~ 13:25:02# wget https://raw.githubusercontent.com/rancher/local-path-provisioner/master/deploy/local-path-storage.yaml
#可以以直接在浏览器搜
# 查看资源配置
root@master30 ~ 13:25:35# vim local-path-storage.yaml
yaml
apiVersion: v1
kind: Namespace
metadata:
name: local-path-storage
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: local-path-provisioner-service-account
namespace: local-path-storage
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: local-path-provisioner-role
namespace: local-path-storage
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list", "watch", "create", "patch", "update", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: local-path-provisioner-role
rules:
- apiGroups: [""]
resources: ["nodes", "persistentvolumeclaims", "configmaps", "pods", "pods/log"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "patch", "update", "delete"]
- apiGroups: [""]
resources: ["events"]
verbs: ["create", "patch"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: local-path-provisioner-bind
namespace: local-path-storage
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: local-path-provisioner-role
subjects:
- kind: ServiceAccount
name: local-path-provisioner-service-account
namespace: local-path-storage
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: local-path-provisioner-bind
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: local-path-provisioner-role
subjects:
- kind: ServiceAccount
name: local-path-provisioner-service-account
namespace: local-path-storage
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: local-path-provisioner
namespace: local-path-storage
spec:
replicas: 1
selector:
matchLabels:
app: local-path-provisioner
template:
metadata:
labels:
app: local-path-provisioner
spec:
serviceAccountName: local-path-provisioner-service-account
containers:
- name: local-path-provisioner
image: rancher/local-path-provisioner:v0.0.35
imagePullPolicy: IfNotPresent
command:
- local-path-provisioner
- --debug
- start
- --config
- /etc/config/config.json
volumeMounts:
- name: config-volume
mountPath: /etc/config/
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: CONFIG_MOUNT_PATH
value: /etc/config/
volumes:
- name: config-volume
configMap:
name: local-path-config
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: local-path
provisioner: rancher.io/local-path
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Delete
---
kind: ConfigMap
apiVersion: v1
metadata:
name: local-path-config
namespace: local-path-storage
data:
config.json: |-
{
"nodePathMap":[
{
"node":"DEFAULT_PATH_FOR_NON_LISTED_NODES",
"paths":["/opt/local-path-provisioner"]
}
]
}
setup: |-
#!/bin/sh
set -eu
mkdir -m 0777 -p "$VOL_DIR"
teardown: |-
#!/bin/sh
set -eu
rm -rf "$VOL_DIR"
helperPod.yaml: |-
apiVersion: v1
kind: Pod
metadata:
name: helper-pod
spec:
priorityClassName: system-node-critical
tolerations:
- key: node.kubernetes.io/disk-pressure
operator: Exists
effect: NoSchedule
containers:
- name: helper-pod
image: busybox
imagePullPolicy: IfNotPresent
local-path-storage.yaml 核心由 5 类资源 组成,每部分各司其职:
| 资源类型 | 名称 | 核心作用 |
|---|---|---|
| Namespace | local-path-storage | 隔离 Local Path Provisioner 相关资源 |
| ConfigMap | local-path-config | 定义本地存储路径 规则(如默认 /opt/local-path-provisioner) |
| ServiceAccount + RBAC | local-path-provisioner | 赋予 Provisioner 操作 PV/PVC 的权限 |
| Deployment | local-path-provisioner | 运行 Local Path Provisioner 核心程序(动态创建本地 PV) |
| StorageClass | local-path | 提供给 PVC 调用的「本地存储模板」 |
bash
# 资源默认部署在命名空间:local-path-storage
root@master30 ~ 13:26:19# kubectl apply -f local-path-storage.yaml
# 确认资源状态
root@master30 ~ 13:27:16# kubectl get sc local-path
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
local-path rancher.io/local-path Delete WaitForFirstConsumer false 67m
root@master30 ~ 13:27:02# kubectl get all -n local-path-storage
NAME READY STATUS RESTARTS AGE
pod/local-path-provisioner-85d9db4886-5t4sn 1/1 Running 0 14s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/local-path-provisioner 1/1 1 1 14s
NAME DESIRED CURRENT READY AGE
replicaset.apps/local-path-provisioner-85d9db4886 1 1 1 14s
验证部署
bash
# 设置默认命名空间
root@master30 ~ 13:34:45# kubectl config set-context --current --namespace default
1. 创建 pvc
bash
#向名为local‑path的存储类申请5Gi本地存储,访问模式为单节点读写,放在 default 命名空间,由 local‑path provisioner 自动生成绑定 PV,供 Pod 挂载持久化数据。
root@master30 ~ 13:35:01# cat > local-path-pvc.yaml <<'EOF'
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: local-path-pvc
namespace: default
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 5Gi
storageClassName: local-path # 指定 Local Path 的 StorageClass
EOF
root@master30 ~ 13:36:28# kubectl apply -f local-path-pvc.yaml
root@master30 ~ 13:37:49# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE
local-path-pvc Pending local-path <unset> 14s
# 等待pod关联才会创建本地目录
2. 创建 deployment
bash
root@master30 ~ 13:38:03# cat > local-path-deployment.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: webapp
name: webapp
namespace: default
spec:
replicas: 2
selector:
matchLabels:
app: webapp
template:
metadata:
labels:
app: webapp
spec:
containers:
- image: nginx
name: nginx
volumeMounts:
- name: webapp-data
mountPath: /usr/share/nginx/html
volumes:
- name: webapp-data
persistentVolumeClaim:
claimName: local-path-pvc
EOF
root@master30 ~ 13:38:18# kubectl apply -f local-path-deployment.yaml
# 查看 pod 状态
root@master30 ~ 13:38:56# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
webapp-6d46b54487-92cpv 1/1 Running 0 30s 10.224.26.130 worker31.ggg.cloud <none> <none>
webapp-6d46b54487-frcvf 1/1 Running 0 30s 10.224.26.191 worker31.ggg.cloud <none> <none>
# 所有节点都调度到 worker31,因为只用worker31可以提供这个卷。
# 查看 pvc 状态
root@master30 ~ 13:38:58# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE
local-path-pvc Bound pvc-756f682b-4e9c-4307-a1af-0e315cb282a7 5Gi RWO local-path <unset> 88s
# 写入测试文件
root@master30 ~ 13:40:24# kubectl exec webapp-6d46b54487-92cpv -- bash -c 'echo Hello World From Nginx > /usr/share/nginx/html/index.html'
#查看本地路径
root@master30 ~ 13:41:24# vim local-path-storage.yaml
....
kind: ConfigMap
....
"paths":["/opt/local-path-provisioner"]
.....
# 验证本地路径
root@worker31 ~ 12:12:53# ls /opt/local-path-provisioner/pvc-756f682b-4e9c-4307-a1af-0e315cb282a7_default_local-path-pvc/
index.html
root@worker31 ~ 13:43:46# cat /opt/local-path-provisioner/pvc-756f682b-4e9c-4307-a1af-0e315cb282a7_default_local-path-pvc/index.html
Hello World From Nginx
3. 清理资源
bash
# 删除 deployment
root@master30 ~ 13:43:28# kubectl delete deployments.apps webapp
# 删除 pvc
root@master30 ~ 13:44:20# kubectl delete pvc local-path-pvc
# 本地目录也被删除
root@worker31 ~ 13:44:00# ls /opt/local-path-provisioner/pvc-756f682b-4e9c-4307-a1af-0e315cb282a7_default_local-path-pvc/
ls: cannot access '/opt/local-path-provisioner/pvc-756f682b-4e9c-4307-a1af-0e315cb282a7_default_local-path-pvc/': No such file or directory
部署 NFS provisioner
部署 NFS 服务
bash
# 安装 NFS server
root@master30 ~ 13:45:35# apt install -y nfs-kernel-server
# 安创建NFS目录 修改创建文件夹的权限
root@master30 ~ 13:45:35# mkdir -m 777 /shares/
# 配置共享,允许所有客户端访问
root@master30 ~ 13:50:25# cat << EOF > /etc/exports
/shares *(rw,sync,no_root_squash,no_all_squash,insecure)
EOF
# 重启 nfs server
root@master30 ~ 13:50:37# systemctl restart nfs-server.service
# 客户端安装
root@worker31 ~ 13:51:03# apt install -y nfs-common
root@worker32 ~ 13:51:07# apt install -y nfs-common
部署 NFS provisioner
NFS没有内置制备器,这里我们自定义外部分配器。
bash
# 资源部署到命名空间:kube-storage
root@master30 ~ 13:50:50# kubectl create ns kube-storage
root@master30 ~ 13:51:33# kubectl config set-context --current --namespace kube-storage
1. 创建 RBAC 权限
创建 nfs-rbac.yaml 文件,赋予 Provisioner 操作 PV/PVC 的权限。
yaml
root@master30:~# cat > nfs-rbac.yaml <<'EOF'
apiVersion: v1
kind: ServiceAccount
metadata:
name: nfs-client-provisioner
namespace: kube-storage
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: nfs-client-provisioner-runner
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["create", "update", "patch"]
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: run-nfs-client-provisioner
subjects:
- kind: ServiceAccount
name: nfs-client-provisioner
namespace: kube-storage
roleRef:
kind: ClusterRole
name: nfs-client-provisioner-runner
apiGroup: rbac.authorization.k8s.io
EOF
bash
root@master30 ~ 13:51:56# kubectl apply -f nfs-rbac.yaml
2. 部署 NFS Provisioner 应用
yaml
root@master30:~# cat > nfs-provisioner.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
name: nfs-client-provisioner
namespace: kube-storage
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: nfs-client-provisioner
template:
metadata:
labels:
app: nfs-client-provisioner
spec:
serviceAccountName: nfs-client-provisioner
volumes:
- name: nfs-client-root
nfs:
server: 10.1.8.30 # 同上 NFS 服务器 IP
path: /shares # 核心:同步改为 /shares
containers:
- name: nfs-client-provisioner
image: registry.k8s.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2
volumeMounts:
- name: nfs-client-root
mountPath: /persistentvolumes
env:
- name: PROVISIONER_NAME
value: fuseim.pri/ifs
- name: NFS_SERVER
value: 10.1.8.30 # 替换为你的 NFS 服务器 IP
- name: NFS_PATH
value: /shares # 核心:共享路径改为 /shares
EOF
bash
# 部署 NFS Provisioner 应用
root@master30 ~ 13:52:48# kubectl apply -f nfs-provisioner.yaml
# 查看 NFS Provisioner 应用
root@master30 ~ 13:53:49# kubectl get deployments.apps -n kube-storage
NAME READY UP-TO-DATE AVAILABLE AGE
nfs-client-provisioner 1/1 1 1 50s
3. 创建 NFS StorageClass
yaml
root@master30:~# cat > nfs-storageclass.yaml <<'EOF'
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: nfs-storage
namespace: kube-storage
provisioner: fuseim.pri/ifs # 必须和 Provisioner 名称一致
parameters:
onDelete: retain # 删除 PVC 保留 NFS 数据
archiveOnDelete: "false"
# 按需设置回收策略
#reclaimPolicy: Retain
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: Immediate
EOF
bash
# 创建 StorageClass
root@master30 ~ 13:54:07# kubectl apply -f nfs-storageclass.yaml
# 查看 StorageClass
root@master30 ~ 13:54:20# kubectl get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
nfs-storage fuseim.pri/ifs Delete Immediate true 7s
验证部署
在default命名空间测试。
bash
root@master30 ~ 13:54:56# kubectl config set-context --current --namespace default
1. 创建 pvc
bash
root@master30 ~ 13:55:19# cat > nfs-pvc.yaml <<'EOF'
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: webclaim
namespace: default
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
storageClassName: "nfs-storage"
EOF
# 创建 pvc
root@master30 ~ 13:55:32# kubectl apply -f nfs-pvc.yaml
# 查看 pvc 状态
root@master30 ~ 13:55:40# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE
webclaim Bound pvc-7c9258e7-7589-4ecf-bce1-85611518a351 5Gi RWO nfs-storage <unset> 12s
# 查看 pv 状态
root@master30 ~ 13:55:52# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS VOLUMEATTRIBUTESCLASS REASON AGE
pvc-7c9258e7-7589-4ecf-bce1-85611518a351 5Gi RWO Delete Bound default/webclaim nfs-storage <unset> 24s
# 查看 nfs 共享目录
root@master30 ~ 13:56:04# ls /shares/
default-webclaim-pvc-7c9258e7-7589-4ecf-bce1-85611518a351
# 准备 web 主页
root@master30 ~ 13:56:33# echo Hello World From Nginx > /shares/default-webclaim-pvc-7c9258e7-7589-4ecf-bce1-85611518a351/index.html
2. 创建 deployment
bash
root@master30 ~ 13:56:42# cat > nfs-deploy.yaml << EOF
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: webapp
name: webapp
spec:
replicas: 2
selector:
matchLabels:
app: webapp
template:
metadata:
labels:
app: webapp
spec:
containers:
- image: nginx
name: nginx
volumeMounts:
- name: webapp-data
mountPath: /usr/share/nginx/html
volumes:
- name: webapp-data
persistentVolumeClaim:
claimName: webclaim
EOF
root@master30 ~ 13:56:52# kubectl apply -f nfs-deploy.yaml
root@master30 ~ 13:57:00# kubectl expose deployment webapp --target-port 80 --port 80 --protocol TCP
root@master30 ~ 13:57:05# kubectl get svc webapp
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
webapp ClusterIP 10.99.50.211 <none> 80/TCP 13s
# 访问测试
root@master30 ~ 13:57:18# curl 10.99.50.211
Hello World From Nginx
root@master30 ~ 13:57:35# curl 10.99.50.211
Hello World From Nginx
3. 清理资源
bash
# 删除 deployment
root@master30 ~ 13:57:36# kubectl delete deployments.apps webapp
# 删除 pvc,pv也将一起删除
root@master30 ~ 13:57:48# kubectl delete pvc webclaim
root@master30 ~ 13:57:59# kubectl get pv
No resources found
# 查看后端存储中数据,仍然保留
root@master30 ~ 13:58:23# ls /shares/default-webclaim-pvc-7c9258e7-7589-4ecf-bce1-85611518a351/
index.html
保留 NFS provisioner ,下一章 statefulset 使用。