目录
- 一.系统环境
- 二.前言
- [三.PodSecurityPolicy 简介](#三.PodSecurityPolicy 简介)
- [四.为什么需要 PodSecurityPolicy](#四.为什么需要 PodSecurityPolicy)
- 五.给客户端授权
- 六.启用PodSecurityPolicy（PSP）
- 七.PSP规则之禁止创建特权用户pod
- 八.PSP规则之数据卷
- [8.1 限定数据卷使用某一个目录](#8.1 限定数据卷使用某一个目录)
- [8.2 限定数据卷的类型为emptyDir](#8.2 限定数据卷的类型为emptyDir)
- 九.PSP规则之指定使用宿主机网络
- 十.PSP规则之pod使用特定的UID运行
- 十一.PSP规则之指定hostport端口
- 十二.PSP规则实战1
- 十三.PSP规则实战2:创建deployment
- 十四.总结
一.系统环境
本文主要基于Kubernetes1.22.2和Linux操作系统Ubuntu 18.04。
| 服务器版本 | docker软件版本 | Kubernetes(k8s)集群版本 | CPU架构 |
|---|---|---|---|
| Ubuntu 18.04.5 LTS | Docker version 20.10.14 | v1.22.2 | x86_64 |
Kubernetes集群架构：k8scludes1作为master节点，k8scludes2，k8scludes3作为worker节点。
| 服务器 | 操作系统版本 | CPU架构 | 进程 | 功能描述 |
|---|---|---|---|---|
| k8scludes1/192.168.110.128 | Ubuntu 18.04.5 LTS | x86_64 | docker，kube-apiserver，etcd，kube-scheduler，kube-controller-manager，kubelet，kube-proxy，coredns，calico | k8s master节点 |
| k8scludes2/192.168.110.129 | Ubuntu 18.04.5 LTS | x86_64 | docker，kubelet，kube-proxy，calico | k8s worker节点 |
| k8scludes3/192.168.110.130 | Ubuntu 18.04.5 LTS | x86_64 | docker，kubelet，kube-proxy，calico | k8s worker节点 |
二.前言
随着容器化和微服务架构的普及，Kubernetes 已经成为企业中编排和管理容器的主要平台。然而，随着集群中运行的容器数量和种类的增加，确保容器的安全性变得越来越重要。PodSecurityPolicy（PSP）是 Kubernetes 用于限制 Pod 创建的一组规则，以确保集群的安全性。在本文中，我们将深入了解 PSP 的概念、重要性以及如何在 Kubernetes 集群中实施。
使用PodSecurityPolicy（PSP）的前提 是已经有一套可以正常运行的Kubernetes集群，关于Kubernetes(k8s)集群的安装部署，可以查看博客《Ubuntu 安装部署Kubernetes(k8s)集群》https://www.cnblogs.com/renshengdezheli/p/17632858.html。
三.PodSecurityPolicy 简介
PodSecurityPolicy 是一个内置的准入控制器， 允许集群管理员控制 Pod 规约中涉及安全的敏感内容。
首先，在集群中创建一个或多个 PodSecurityPolicy 资源来定义 Pod 必须满足的要求。 然后，创建 RBAC 规则来决定为特定的 Pod 应用哪个 PodSecurityPolicy。 如果 Pod 满足其 PSP 的要求，则照常被允许进入集群。 在某些情况下，PSP 还可以修改 Pod 字段，有效地为这些字段创建新的默认值。 如果 Pod 不符合 PSP 要求，则被拒绝进入集群，并且无法运行。
简单说来，就是PodSecurityPolicy 可以设置pod里的进程只能以某个UID运行，或者设置pod只能使用某一类型存储，不能使用其他，比如只能使用nfs存储。
四.为什么需要 PodSecurityPolicy
在 Kubernetes 中，我们定义了 Deployment、StatefulSet 和 Service 等资源。 这些资源代表软件应用程序的构建块。Kubernetes 集群中的各种控制器根据这些资源做出反应， 创建更多的 Kubernetes 资源或配置一些软件或硬件来实现我们的目标。
在大多数 Kubernetes 集群中，由 RBAC（基于角色的访问控制）规则 控制对这些资源的访问。 list、get、create、edit 和 delete 是 RBAC 关心的 API 操作类型， 但 RBAC 不考虑其所控制的资源中加入了哪些设置。例如， Pod 几乎可以是任何东西，例如简单的网络服务器，或者是特权命令提示（提供对底层服务器节点和所有数据的完全访问权限）。 这对 RBAC 来说都是一样的：Pod 就是 Pod 而已。
要控制集群中定义的资源允许哪些类型的设置，除了 RBAC 之外，还需要准入控制。 从 Kubernetes 1.3 开始，内置 PodSecurityPolicy 一直被作为 Pod 安全相关字段的准入控制机制。 使用 PodSecurityPolicy，可以防止"创建 Pod"这个能力自动变成"每个集群节点上的 root 用户"， 并且无需部署额外的外部准入控制器。
五.给客户端授权
创建minsvcbug目录存放配置文件。
shell
root@k8scludes1:~# mkdir minsvcbug
root@k8scludes1:~# cd minsvcbug
root@k8scludes1:~/minsvcbug# pwd
/root/minsvcbug创建minsvcbug命名空间。
shell
root@k8scludes1:~/minsvcbug# kubectl create ns minsvcbug
namespace/minsvcbug created切换命名空间。
shell
root@k8scludes1:~/minsvcbug# kubens minsvcbug
Context "kubernetes-admin@kubernetes" modified.
Active namespace is "minsvcbug".
root@k8scludes1:~/minsvcbug# kubectl get pod
No resources found in minsvcbug namespace.本次使用etcd2机器作为k8s客户端，kctom为kubeconfig文件，里面的用户为tom,需要给tom授权，客户端才能具有相关访问权限。
关于kubeconfig文件的详细操作，请查看博客《Kubernetes(k8s)访问控制：身份认证》。
关于授权的详细操作，请查看博客《Kubernetes(k8s)访问控制：权限管理之RBAC鉴权》。
shell
[root@etcd2 ~]# ls
kctom 查看minsvcbug命名空间下的pod，现在tom用户还没有minsvcbug命名空间的pod访问权限。
shell
[root@etcd2 ~]# kubectl get pod --kubeconfig=kctom -n minsvcbug
Error from server (Forbidden): pods is forbidden: User "tom" cannot list resource "pods" in API group "" in the namespace "minsvcbug"给客户端tom用户授予创建/删除/查看pod的权限，编辑角色role配置文件。
yaml
root@k8scludes1:~/minsvcbug# vim role1.yaml
#role1角色对pod,svc,secret具有get,list,watch,create,delete权限
root@k8scludes1:~/minsvcbug# cat role1.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
creationTimestamp: null
name: role1
rules:
- apiGroups:
- ""
resources:
- pods
- services
- secrets
verbs:
- get
- list
- watch
- create
- delete创建角色role1。
shell
root@k8scludes1:~/minsvcbug# kubectl apply -f role1.yaml
role.rbac.authorization.k8s.io/role1 created查看角色。
shell
root@k8scludes1:~/minsvcbug# kubectl get role -o wide
NAME CREATED AT
role1 2022-05-20T03:42:48Z查看role1的详细描述。
shell
root@k8scludes1:~/minsvcbug# kubectl describe role role1
Name: role1
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
pods [] [] [get list watch create delete]
secrets [] [] [get list watch create delete]
services [] [] [get list watch create delete]把角色role1绑定给tom用户。
shell
root@k8scludes1:~/minsvcbug# kubectl create rolebinding rolebindingtotom --role=role1 --user=tom
rolebinding.rbac.authorization.k8s.io/rolebindingtotom created
root@k8scludes1:~/minsvcbug# kubectl get rolebinding
NAME ROLE AGE
rolebindingtotom Role/role1 11s给tom用户授予创建/删除/查看pod的权限之后，在客户端查看。可以看到tom用户对minsvcbug命名空间的pod有访问权限了。
shell
[root@etcd2 ~]# kubectl get pod --kubeconfig=kctom -n minsvcbug
No resources found in minsvcbug namespace.可以看到tom用户对default命名空间的pod没有访问权限。
shell
[root@etcd2 ~]# kubectl get pod --kubeconfig=kctom
Error from server (Forbidden): pods is forbidden: User "tom" cannot list resource "pods" in API group "" in the namespace "default"在客户端etcd2机器上创建目录存放文件。
shell
[root@etcd2 ~]# mkdir minsvcbug
[root@etcd2 ~]# cd minsvcbug/
[root@etcd2 minsvcbug]# pwd
/root/minsvcbug编辑pod配置文件，使用nginx创建pod。
yaml
[root@etcd2 minsvcbug]# cat pod.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
#当需要关闭容器时，立即杀死容器而不等待默认的30秒优雅停机时长。
terminationGracePeriodSeconds: 0
containers:
- image: hub.c.163.com/library/nginx:latest
#imagePullPolicy: IfNotPresent：表示如果本地已经存在该镜像，则不重新下载；否则从远程 Docker Hub 下载该镜像
imagePullPolicy: IfNotPresent
name: podtest
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}在客户端使用nginx镜像创建pod。
shell
[root@etcd2 minsvcbug]# kubectl apply -f pod.yaml --kubeconfig=kctom -n minsvcbug
pod/podtest createdpod创建成功，说明成功授权了。
shell
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 36s 10.244.218.132 k8scludes2 <none> <none>
[root@etcd2 minsvcbug]# kubectl delete -f pod.yaml --kubeconfig=kctom -n minsvcbug
pod "podtest" deleted现在tom用户就具备了minsvcbug命名空间的创建/删除/查看pod的权限了。
shell
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
No resources found in minsvcbug namespace.六.启用PodSecurityPolicy（PSP）
接下来启用PSP，编辑kube-apiserver.yaml文件。
shell
root@k8scludes1:~/minsvcbug# ls /etc/kubernetes/manifests/kube-apiserver.yaml
/etc/kubernetes/manifests/kube-apiserver.yaml启用PSP需要添加- --enable-admission-plugins=PodSecurityPolicy。
shell
root@k8scludes1:~/minsvcbug# vim /etc/kubernetes/manifests/kube-apiserver.yaml
root@k8scludes1:~/minsvcbug# grep enable-admission-plugins /etc/kubernetes/manifests/kube-apiserver.yaml
- --enable-admission-plugins=NodeRestriction,PodSecurityPolicy重启kubelet使配置生效。
shell
root@k8scludes1:~/minsvcbug# systemctl restart kubelet
root@k8scludes1:~/minsvcbug# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8scludes1 Ready control-plane,master 33d v1.22.2
k8scludes2 Ready <none> 33d v1.22.2
k8scludes3 Ready <none> 33d v1.22.2PSP(pod security policy)默认没有开启，如果开启PSP之后，没有写PSP规则的话，任何用户都创建不了POD，包括管理员。
注意：一定要等k8s正常工作之后再启动PSP，如果etcd,api都没启动好就启动PSP，则相关pod都启动不了了 ，等PSP启动之后再去写PSP规则。
启用PSP之后，创建pod必须先访问PSP规则，通过了PSP规则才能创建POD，否则创建pod失败。
默认情况下，普通用户没有访问PSP规则的权限，管理员用户可以直接访问PSP规则，创建好PSP规则之后，对普通用户要做一个RBAC的权限设置，使其可以访问某个PSP规则，再决定是不是能够把pod创建出 来(1.是否有足够的RBAC权限 2.创建的POD是否符合PSP规则)。
七.PSP规则之禁止创建特权用户pod
查看PSP规则。
shell
root@k8scludes1:~/minsvcbug# kubectl get psp
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir如果没有任何PSP规则，则任何用户都不能创建pod。
编辑PSP规则，我们的目的是拒绝特权pod运行，privileged: false 表示不允许特权pod运行。
yaml
root@k8scludes1:~/minsvcbug# vim psp-deny-privileges-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-deny-privileges-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-deny-privileges-pod
spec:
# Don't allow privileged pods!
privileged: false
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
volumes:
- '*'创建psp。
shell
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-deny-privileges-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-deny-privileges-pod createdpsp-deny-privileges-pod规则就创建好了。
shell
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-deny-privileges-pod false RunAsAny RunAsAny RunAsAny RunAsAny false *
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir创建psp-deny-privileges-pod的PSP规则之后,在客户端使用普通用户创建pod。
编辑pod配置文件。
yaml
[root@etcd2 minsvcbug]# cat pod.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
#当需要关闭容器时，立即杀死容器而不等待默认的30秒优雅停机时长。
terminationGracePeriodSeconds: 0
containers:
- image: hub.c.163.com/library/nginx:latest
#imagePullPolicy: IfNotPresent：表示如果本地已经存在该镜像，则不重新下载；否则从远程 Docker Hub 下载该镜像
imagePullPolicy: IfNotPresent
name: podtest
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}创建pod失败，就算普通用户tom创建没有特权用户的pod，因为tom没有访问PSP规则的权限，所以创建pod失败。
shell
[root@etcd2 minsvcbug]# kubectl apply -f pod.yaml --kubeconfig=kctom -n minsvcbug
Error from server (Forbidden): error when creating "pod.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: []
[root@etcd2 minsvcbug]# kubectl get pod --kubeconfig=kctom -n minsvcbug
No resources found in minsvcbug namespace.下面要给tom用户授予PSP的访问权限，对PSP的授权可以是clusterrole，也可以是role，PSP规则是全局生效的，在其他命名空间也可以看到，本次使用clusterrole授权。
生成clusterrole-access-psp-rule的yaml文件，表示：角色clusterrole-access-psp-rule对名为psp-deny-privileges-pod有use权限。
shell
root@k8scludes1:~# kubectl create clusterrole clusterrole-access-psp-rule --verb=use --resource=psp --resource-name=psp-deny-privileges-pod --dry-run=client -o yaml >clusterrole-access-psp-rule.yaml生成的yaml文件如下：
yaml
root@k8scludes1:~# vim clusterrole-access-psp-rule.yaml
root@k8scludes1:~# cat clusterrole-access-psp-rule.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
creationTimestamp: null
name: clusterrole-access-psp-rule
rules:
- apiGroups:
- policy
resourceNames:
- psp-deny-privileges-pod
resources:
- podsecuritypolicies
verbs:
- use创建clusterrole-access-psp-rule。
shell
root@k8scludes1:~# kubectl apply -f clusterrole-access-psp-rule.yaml
clusterrole.rbac.authorization.k8s.io/clusterrole-access-psp-rule created
root@k8scludes1:~# kubectl get clusterrole | grep clusterrole-access-psp-rule
clusterrole-access-psp-rule 2022-05-20T10:21:15Z查看角色的详细描述。
shell
root@k8scludes1:~# kubectl describe clusterrole clusterrole-access-psp-rule
Name: clusterrole-access-psp-rule
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
podsecuritypolicies.policy [] [psp-deny-privileges-pod] [use]把角色clusterrole-access-psp-rule绑定给tom用户。
shell
root@k8scludes1:~# kubectl create rolebinding rolebinding-clusterrole-access-psp --clusterrole=clusterrole-access-psp-rule --user=tom
rolebinding.rbac.authorization.k8s.io/rolebinding-clusterrole-access-psp created
root@k8scludes1:~# kubectl get rolebinding -o wide
NAME ROLE AGE USERS GROUPS SERVICEACCOUNTS
rolebinding-clusterrole-access-psp ClusterRole/clusterrole-access-psp-rule 15s tom
rolebindingtotom Role/role1 6h40m tom 给tom用户授予PSP的访问权限之后，到客户端创建无特权用户的pod。
shell
[root@etcd2 minsvcbug]# kubectl apply -f pod.yaml --kubeconfig=kctom -n minsvcbug
pod/podtest createdpod创建成功，现在tom用户就可以创建无特权用户的pod了。
shell
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 21s 10.244.218.145 k8scludes2 <none> <none>删除pod。
shell
[root@etcd2 minsvcbug]# kubectl delete pod podtest --kubeconfig=kctom -n minsvcbug
pod "podtest" deleted
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
No resources found in minsvcbug namespace.编辑pod文件，securityContext:privileged: true 表示创建特权用户的pod。
yaml
[root@etcd2 minsvcbug]# cat podprivileged.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
#当需要关闭容器时，立即杀死容器而不等待默认的30秒优雅停机时长。
terminationGracePeriodSeconds: 0
containers:
- image: hub.c.163.com/library/nginx:latest
#imagePullPolicy: IfNotPresent：表示如果本地已经存在该镜像，则不重新下载；否则从远程 Docker Hub 下载该镜像
imagePullPolicy: IfNotPresent
name: podtest
#创建特权pod
securityContext:
privileged: true
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}创建pod失败，Privileged containers are not allowed 创建特权容器是不允许的。
shell
[root@etcd2 minsvcbug]# kubectl apply -f podprivileged.yaml --kubeconfig=kctom -n minsvcbug
Error from server (Forbidden): error when creating "podprivileged.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
No resources found in minsvcbug namespace.删除psp-deny-privileges-pod规则。
shell
root@k8scludes1:~/minsvcbug# kubectl delete psp psp-deny-privileges-pod
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy "psp-deny-privileges-pod" deleted
root@k8scludes1:~/minsvcbug# kubectl get psp
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir删除clusterrole-access-psp-rule(不删除也可以)。
shell
root@k8scludes1:~/minsvcbug# kubectl get clusterrole | grep clusterrole-access-psp-rule
clusterrole-access-psp-rule 2022-05-20T10:21:15Z
root@k8scludes1:~/minsvcbug# kubectl delete clusterrole clusterrole-access-psp-rule
clusterrole.rbac.authorization.k8s.io "clusterrole-access-psp-rule" deleted删除rolebinding-clusterrole-access-psp(不删除也可以)。
shell
root@k8scludes1:~/minsvcbug# kubectl delete rolebinding rolebinding-clusterrole-access-psp
rolebinding.rbac.authorization.k8s.io "rolebinding-clusterrole-access-psp" deleted总结：
- 在启用了PSP之后，没有任何PSP规则的情况下，任何用户(包括管理员)都不能创建pod；
- 在启用了PSP之后，且创建了PSP规则，要确保用户能访问到此PSP规则，创建clusterrole，让用户能use某条PSP规则；
- 能不能成功创建pod，决定权在于是否具备创建pod的RBAC权限，有对应的RBAC权限之后，创建的pod是否满足PSP规则。
八.PSP规则之数据卷
8.1 限定数据卷使用某一个目录
编辑psp配置文件，privileged: true表示可以创建特权用户的pod，volumes:- '*' 表示可以使用所有类型的数据卷，allowedHostPaths:- pathPrefix: "/tmp" 表示数据卷只能挂载/tmp目录，psp-mount-pod这条PSP规则规定hostPath只能使用/tmp目录。
yaml
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
volumes:
- '*'
allowedHostPaths:
- pathPrefix: "/tmp"创建PSP。
shell
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod created
root@k8scludes1:~/minsvcbug# kubectl get psp
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false *
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir创建了新的PSP规则之后，需要重新给客户端的tom用户授权。
创建clusterrole--psp-mount-rule，角色clusterrole--psp-mount-rule对名为psp-mount-pod的psp规则有use权限。
shell
root@k8scludes1:~/minsvcbug# kubectl create clusterrole clusterrole--psp-mount-rule --verb=use --resource=psp --resource-name=psp-mount-pod
clusterrole.rbac.authorization.k8s.io/clusterrole--psp-mount-rule created把角色clusterrole--psp-mount-rule绑定给tom用户。
shell
root@k8scludes1:~/minsvcbug# kubectl create rolebinding rolebinding-clusterrole-mount-psp --clusterrole=clusterrole--psp-mount-rule --user=tom
rolebinding.rbac.authorization.k8s.io/rolebinding-clusterrole-mount-psp created给客户端tom用户授权之后，编辑pod配置文件，volumes:- name: v1 hostPath: path: /opt 表示创建名叫v1的数据卷，目录为/opt ,把v1这个数据卷挂载到容器的/data目录。securityContext:privileged: true表示创建特权pod。
关于数据卷的详细操作，请查看博客《Kubernetes(k8s)存储管理之数据卷volumes(一)：volumes的引入和emptyDir数据卷》。
yaml
[root@etcd2 minsvcbug]# vim podmount.yaml
[root@etcd2 minsvcbug]# cat podmount.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
#当需要关闭容器时，立即杀死容器而不等待默认的30秒优雅停机时长。
terminationGracePeriodSeconds: 0
#数据卷v1
volumes:
- name: v1
hostPath:
path: /opt
containers:
- image: hub.c.163.com/library/nginx:latest
#imagePullPolicy: IfNotPresent：表示如果本地已经存在该镜像，则不重新下载；否则从远程 Docker Hub 下载该镜像
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
#把数据卷v1挂载到/data目录
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}创建pod失败，"/opt": is not allowed to be used] ，/opt目录不被允许使用。
shell
[root@etcd2 minsvcbug]# kubectl apply -f podmount.yaml --kubeconfig=kctom -n minsvcbug
Error from server (Forbidden): error when creating "podmount.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.volumes[0].hostPath.pathPrefix: Invalid value: "/opt": is not allowed to be used]修改pod配置文件，把v1数据卷的/opt目录换成/tmp目录。
yaml
[root@etcd2 minsvcbug]# vim podmount.yaml
[root@etcd2 minsvcbug]# cat podmount.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
#当需要关闭容器时，立即杀死容器而不等待默认的30秒优雅停机时长。
terminationGracePeriodSeconds: 0
volumes:
- name: v1
hostPath:
path: /tmp
containers:
- image: hub.c.163.com/library/nginx:latest
#imagePullPolicy: IfNotPresent：表示如果本地已经存在该镜像，则不重新下载；否则从远程 Docker Hub 下载该镜像
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}创建pod。
shell
[root@etcd2 minsvcbug]# kubectl apply -f podmount.yaml --kubeconfig=kctom -n minsvcbug
pod/podtest created把v1数据卷的/opt目录换成/tmp目录之后，pod就创建成功了。
shell
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 14s 10.244.218.134 k8scludes2 <none> <none>
[root@etcd2 minsvcbug]# kubectl delete pod podtest --kubeconfig=kctom -n minsvcbug
pod "podtest" deleted
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
No resources found in minsvcbug namespace.8.2 限定数据卷的类型为emptyDir
编辑PSP配置文件，volumes:- 'emptyDir' 限定数据卷的类型为emptyDir。
yaml
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
volumes:
- 'emptyDir'
#allowedHostPaths:
#- pathPrefix: "/tmp"创建PSP。
shell
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod configured
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false emptyDir
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir编辑pod配置文件，hostPath:path: /tmp 指定hostPath目录，这里使用的是hostPath类型的数据卷。
yaml
root@k8scludes1:~/minsvcbug# cat podmount.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
hostPath:
path: /tmp
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}创建pod失败，Invalid value: "hostPath": hostPath volumes are not allowed to be used 表示不能使用hostPath数据卷。
shell
root@k8scludes1:~/minsvcbug# kubectl apply -f podmount.yaml
Error from server (Forbidden): error when creating "podmount.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.volumes[0]: Invalid value: "hostPath": hostPath volumes are not allowed to be used spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed spec.volumes[0]: Invalid value: "hostPath": hostPath volumes are not allowed to be used spec.volumes[1]: Invalid value: "projected": projected volumes are not allowed to be used spec.volumes[0]: Invalid value: "hostPath": hostPath volumes are not allowed to be used]这次在客户端创建pod。
修改pod配置文件，emptyDir: {}表示使用emptyDir数据卷。
yaml
[root@etcd2 minsvcbug]# cat podmount.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}创建pod失败，pod的数据卷类型修改为emptyDir之后，还是报错：Invalid value: "projected": projected volumes are not allowed to be used]。
shell
[root@etcd2 minsvcbug]# kubectl apply -f podmount.yaml --kubeconfig=kctom -n minsvcbug
Error from server (Forbidden): error when creating "podmount.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.volumes[1]: Invalid value: "projected": projected volumes are not allowed to be used]先修改PSP规则，数据卷类型还是设置为所有类型volumes:- "*" 。
yaml
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
volumes:
- "*"
#- 'emptyDir'
#allowedHostPaths:
#- pathPrefix: "/tmp"应用PSP。
shell
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod configured
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false *
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDirPSP规则的数据卷类型修改为所有类型之后，创建pod成功。
shell
root@k8scludes1:~/minsvcbug# cat podmount.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
root@k8scludes1:~/minsvcbug# kubectl apply -f podmount.yaml
pod/podtest created
root@k8scludes1:~/minsvcbug# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 9s 10.244.218.155 k8scludes2 <none> <none>使用yaml文件的格式查看podtest，可以发现有两个数据卷，一个是emptyDir类型的，一个是projected类型的。
yaml
root@k8scludes1:~/minsvcbug# kubectl get pod podtest -o yaml
apiVersion: v1
kind: Pod
metadata:
......
volumes:
- emptyDir: {}
name: v1
- name: kube-api-access-8xjp8
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace
......更新软件源。
shell
root@k8scludes1:~/minsvcbug# apt-get update安装jq,jq可以使JSON格式更好看。
shell
root@k8scludes1:~/minsvcbug# apt-get install jq -y
root@k8scludes1:~/minsvcbug# which jq
/usr/bin/jq以json格式的方式查看podtest，jsonpath='{.spec.volumes}'表示只查看spec字段下的volumes，这样显示不直观，使用jq使JSON格式化。
json
root@k8scludes1:~/minsvcbug# kubectl get pod podtest -o jsonpath='{.spec.volumes}'
[{"emptyDir":{},"name":"v1"},{"name":"kube-api-access-8xjp8","projected":{"defaultMode":420,"sources":[{"serviceAccountToken":{"expirationSeconds":3607,"path":"token"}},{"configMap":{"items":[{"key":"ca.crt","path":"ca.crt"}],"name":"kube-root-ca.crt"}},{"downwardAPI":{"items":[{"fieldRef":{"apiVersion":"v1","fieldPath":"metadata.namespace"},"path":"namespace"}]}}]}}]root@k8scludes1:~/minsvcbug#可以发现不仅有emptyDir数据卷，还有projected类型的数据卷。
json
root@k8scludes1:~/minsvcbug# kubectl get pod podtest -o jsonpath='{.spec.volumes}' | jq
[
{
"emptyDir": {},
"name": "v1"
},
{
"name": "kube-api-access-8xjp8",
"projected": {
"defaultMode": 420,
"sources": [
{
"serviceAccountToken": {
"expirationSeconds": 3607,
"path": "token"
}
},
{
"configMap": {
"items": [
{
"key": "ca.crt",
"path": "ca.crt"
}
],
"name": "kube-root-ca.crt"
}
},
{
"downwardAPI": {
"items": [
{
"fieldRef": {
"apiVersion": "v1",
"fieldPath": "metadata.namespace"
},
"path": "namespace"
}
]
}
}
]
}
}
]所以创建PSP规则的时候不仅要指定emptyDir类型的数据卷，还要指定projected类型的数据卷，修改PSP规则。
yaml
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
volumes:
- 'emptyDir'
- 'projected'
#allowedHostPaths:
#- pathPrefix: "/tmp"应用PSP规则。
shell
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod configured
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false emptyDir,projected
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir删除pod。
shell
root@k8scludes1:~/minsvcbug# kubectl get pod
NAME READY STATUS RESTARTS AGE
podtest 1/1 Running 0 13m
root@k8scludes1:~/minsvcbug# kubectl delete pod podtest
pod "podtest" deleted修改pod配置文件，数据卷类型为emptyDir。
yaml
[root@etcd2 minsvcbug]# cat podmount.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}客户端继续创建pod，在客户端成功创建emptyDir类型数据卷的pod。
shell
[root@etcd2 minsvcbug]# kubectl apply -f podmount.yaml --kubeconfig=kctom -n minsvcbug
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 6s 10.244.218.136 k8scludes2 <none> <none>删除pod。
shell
[root@etcd2 minsvcbug]# kubectl delete pod podtest --kubeconfig=kctom -n minsvcbug
pod "podtest" deleted
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
No resources found in minsvcbug namespace.九.PSP规则之指定使用宿主机网络
编辑pod配置文件，hostNetwork: true 表示是否使用宿主机网络，true表示使用宿主机网络，默认是false，不使用宿主机网络。
hostNetwork: true相当于docker run -dit --name=nginx --network=net nginx 共享宿主机网络空间。
yaml
[root@etcd2 minsvcbug]# vim podnetwork.yaml
[root@etcd2 minsvcbug]# cat podnetwork.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
#hostNetwork: true 表示pod使用宿主机网络
hostNetwork: true
restartPolicy: Always
status: {}创建pod失败，Invalid value: true: Host network is not allowed to be used]表示 pod不能使用宿主机网络。
shell
[root@etcd2 minsvcbug]# kubectl apply -f podnetwork.yaml --kubeconfig=kctom -n minsvcbug
Error from server (Forbidden): error when creating "podnetwork.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.securityContext.hostNetwork: Invalid value: true: Host network is not allowed to be used]修改PSP规则，使pod可以使用宿主机网络，hostNetwork: true 表示可以使用宿主机网络。
yaml
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
hostNetwork: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
volumes:
- 'emptyDir'
- 'projected'
#allowedHostPaths:
#- pathPrefix: "/tmp"应用PSP规则。
shell
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod configured
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false emptyDir,projected
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir客户端再次创建使用宿主机网络的pod，这次pod创建成功。
shell
[root@etcd2 minsvcbug]# kubectl apply -f podnetwork.yaml --kubeconfig=kctom -n minsvcbug
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 17s 192.168.110.129 k8scludes2 <none> <none>删除pod。
shell
[root@etcd2 minsvcbug]# kubectl delete pod podtest --kubeconfig=kctom -n minsvcbug
pod "podtest" deleted在default命名空间，创建宿主机网络的pod。
shell
root@k8scludes1:~/minsvcbug# kubectl apply -f podnetwork.yaml
pod/podtest created
root@k8scludes1:~/minsvcbug# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 7s 192.168.110.129 k8scludes2 <none> <none>进入pod。
shell
#可以看到pod里使用的是宿主机里的网络
root@k8scludes1:~/minsvcbug# kubectl exec -it podtest -- bash
root@k8scludes2:/# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:0c:29:71:45:c1 brd ff:ff:ff:ff:ff:ff
inet 192.168.110.129/24 brd 192.168.110.255 scope global ens32
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe71:45c1/64 scope link
valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:fd:bc:7b:4b brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
6: tunl0@NONE: <NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue state UNKNOWN group default qlen 1000
link/ipip 0.0.0.0 brd 0.0.0.0
inet 10.244.218.128/32 scope global tunl0
valid_lft forever preferred_lft forever
#退出pod
root@k8scludes2:/# exit
exit删除pod。
shell
root@k8scludes1:~/minsvcbug# kubectl delete pod podtest
pod "podtest" deleted
root@k8scludes1:~/minsvcbug# kubectl get pod
No resources found in minsvcbug namespace.十.PSP规则之pod使用特定的UID运行
编辑pod配置文件，hostNetwork: true 表示pod使用宿主机网络 。
yaml
root@k8scludes1:~/minsvcbug# cat podnetwork.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
hostNetwork: true
restartPolicy: Always
status: {}创建pod。
shell
root@k8scludes1:~/minsvcbug# kubectl apply -f podnetwork.yaml
pod/podtest created
root@k8scludes1:~/minsvcbug# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 12s 192.168.110.129 k8scludes2 <none> <none>进入pod，可以发现pod默认是以root身份运行的，可不可以设置pod以其他身份运行呢？
shell
root@k8scludes1:~/minsvcbug# kubectl exec -it podtest -- bash
#查看网络，pod使用的是宿主机的网络
root@k8scludes2:/# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:0c:29:71:45:c1 brd ff:ff:ff:ff:ff:ff
inet 192.168.110.129/24 brd 192.168.110.255 scope global ens32
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe71:45c1/64 scope link
valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:4c:05:bc:65 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
6: tunl0@NONE: <NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue state UNKNOWN group default qlen 1000
link/ipip 0.0.0.0 brd 0.0.0.0
inet 10.244.218.128/32 scope global tunl0
valid_lft forever preferred_lft forever
#pod是以root用户运行的
root@k8scludes2:/# id
uid=0(root) gid=0(root) groups=0(root)
#退出pod
root@k8scludes2:/# exit
exit查看PSP规则。
shell
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false emptyDir,projected
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir编辑PSP规则，runAsUser:rule: RunAsAny 中 runAsUser表示以某种身份运行，RunAsAny表示可以以任何身份运行，rule: MustRunAs ranges:- min: 1000 max: 2000 表示pod必须以1000-2000的UID身份运行。
yaml
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
hostNetwork: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
#rule: RunAsAny
rule: MustRunAs
#MustRunAs 表示pod必须以1000-2000的UID身份运行
ranges:
- min: 1000
max: 2000
fsGroup:
rule: RunAsAny
volumes:
- 'emptyDir'
- 'projected'
#allowedHostPaths:
#- pathPrefix: "/tmp"使PSP规则生效。
shell
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod configured
root@k8scludes1:~/minsvcbug# kubectl get psp
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny MustRunAs RunAsAny RunAsAny false emptyDir,projected
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir因为客户端etcd2需要进入到pod里，以及查看pod的日志，再给客户端添加pods/exec， pods/log权限，直接在Role配置文件中加入即可。
yaml
root@k8scludes1:~/minsvcbug# vim role1.yaml
root@k8scludes1:~/minsvcbug# cat role1.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
creationTimestamp: null
name: role1
rules:
- apiGroups:
- ""
resources:
- pods
#pods/exec， pods/log是新加的权限
- pods/exec
- pods/log
- services
- secrets
verbs:
- get
- list
- watch
- create
- delete应用Role。
shell
root@k8scludes1:~/minsvcbug# kubectl apply -f role1.yaml
role.rbac.authorization.k8s.io/role1 configured查看role1的权限信息。
shell
root@k8scludes1:~/minsvcbug# kubectl describe role role1
Name: role1
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
pods/exec [] [] [get list watch create delete]
pods/log [] [] [get list watch create delete]
pods [] [] [get list watch create delete]
secrets [] [] [get list watch create delete]
services [] [] [get list watch create delete]接下来在客户端etcd2机器进行测试。
把kubeconfig文件kctom复制到~/.kube/config，这样执行命令行就不用每次都指定--kubeconfig了。
shell
[root@etcd2 minsvcbug]# cp kctom ~/.kube/config
[root@etcd2 minsvcbug]# ls ~/.kube/
cache config
[root@etcd2 minsvcbug]# kubectl get pod -n minsvcbug
No resources found in minsvcbug namespace.编辑pod文件。
yaml
[root@etcd2 minsvcbug]# vim poduid.yaml
[root@etcd2 minsvcbug]# cat poduid.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
#当需要关闭容器时，立即杀死容器而不等待默认的30秒优雅停机时长。
terminationGracePeriodSeconds: 0
#emptyDir类型的数据卷
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
#imagePullPolicy: IfNotPresent：表示如果本地已经存在该镜像，则不重新下载；否则从远程 Docker Hub 下载该镜像
imagePullPolicy: IfNotPresent
name: podtest
#创建特权pod
securityContext:
privileged: true
resources: {}
#数据卷挂载
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
#hostNetwork: true 表示pod使用宿主机网络
hostNetwork: true
restartPolicy: Always
status: {}创建pod。
shell
[root@etcd2 minsvcbug]# kubectl apply -f poduid.yaml -n minsvcbug
pod/podtest createdpod创建失败。
shell
[root@etcd2 minsvcbug]# kubectl get pod -n minsvcbug
NAME READY STATUS RESTARTS AGE
podtest 0/1 CrashLoopBackOff 1 (5s ago) 6s查看pod日志：podtest里不允许root来运行，podtest默认运行的是nginx进程，nginx进程需要使用root身份来运行。
shell
[root@etcd2 minsvcbug]# kubectl logs podtest -n minsvcbug
2022/05/24 07:53:04 [warn] 1#1: the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:2
nginx: [warn] the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:2
2022/05/24 07:53:04 [emerg] 1#1: mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)
nginx: [emerg] mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)删除pod。
shell
[root@etcd2 minsvcbug]# kubectl delete pod podtest -n minsvcbug
pod "podtest" deleted编辑pod配置文件，这次pod不运行nginx的默认进程，因为nginx进程需要root才能正常运行，我们运行sleep 100000命令，sleep 100000命令什么用户都可以运行。
yaml
[root@etcd2 minsvcbug]# vim poduid.yaml
[root@etcd2 minsvcbug]# cat poduid.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
#运行sleep 100000命令
command: ["sh","-c","sleep 100000"]
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
hostNetwork: true
restartPolicy: Always
status: {}创建pod。
shell
[root@etcd2 minsvcbug]# kubectl apply -f poduid.yaml -n minsvcbug
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -o wide -n minsvcbug
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 12s 192.168.110.129 k8scludes2 <none> <none>进入pod。
shell
[root@etcd2 minsvcbug]# kubectl exec -it podtest -n minsvcbug -- bash
#现在默认的用户UID为1000
I have no name!@k8scludes2:/$ id
uid=1000 gid=0(root) groups=0(root)
#退出pod
I have no name!@k8scludes2:/$ exit
exit删除pod。
shell
[root@etcd2 minsvcbug]# kubectl delete pod podtest -n minsvcbug
pod "podtest" deleted
[root@etcd2 minsvcbug]# kubectl get pod -n minsvcbug
No resources found in minsvcbug namespace.编辑pod配置文件，securityContext:runAsUser: 3000 表示pod以UID为3000的身份运行。
yaml
[root@etcd2 minsvcbug]# vim poduid.yaml
[root@etcd2 minsvcbug]# cat poduid.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
securityContext:
runAsUser: 3000
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
command: ["sh","-c","sleep 100000"]
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}创建pod失败，UID 3000不在PSP规则规定的{1000 2000}之间，所以创建pod失败。
shell
[root@etcd2 minsvcbug]# kubectl apply -f poduid.yaml -n minsvcbug
Error from server (Forbidden): error when creating "poduid.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.runAsUser: Invalid value: 3000: must be in the ranges: [{1000 2000}]]编辑pod配置文件，securityContext:runAsUser: 1500 表示pod以UID为1500的身份运行。
yaml
[root@etcd2 minsvcbug]# vim poduid.yaml
[root@etcd2 minsvcbug]# cat poduid.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
securityContext:
runAsUser: 1500
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
command: ["sh","-c","sleep 100000"]
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}创建pod成功。
shell
[root@etcd2 minsvcbug]# kubectl apply -f poduid.yaml -n minsvcbug
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -n minsvcbug
NAME READY STATUS RESTARTS AGE
podtest 1/1 Running 0 12s进入pod。
shell
[root@etcd2 minsvcbug]# kubectl exec -it podtest -n minsvcbug -- bash
#UID 1500在{1000 2000}范围，所以pod运行成功，pod运行进程UID也为1500
I have no name!@podtest:/$ id
uid=1500 gid=0(root) groups=0(root)
#退出pod
I have no name!@podtest:/$ exit
exit十一.PSP规则之指定hostport端口
修改一下vim设置，设置为粘贴模式之后，粘贴格式看着更规范。
shell
root@k8scludes1:~/minsvcbug# vim /etc/vim/vimrc
root@k8scludes1:~/minsvcbug# tail -1 /etc/vim/vimrc
set paste编辑PSP规则，因为nginx进程需要root身份才能运行，RUNASUSER 变为RunAsAny，RunAsAny表示可以以任何身份运行。
yaml
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
#hostNetwork: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
#rule: MustRunAs
#ranges:
#- min: 1000
# max: 2000
fsGroup:
rule: RunAsAny
volumes:
- '*'
#- 'emptyDir'
#- 'projected'
#allowedHostPaths:
#- pathPrefix: "/tmp"应用PSP规则。
shell
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod configured
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false *
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir去客户端创建pod。
编辑pod配置文件，containerPort: 80 表示容器端口80，hostPort: 8080表示宿主机端口8080 ，表示把nginx容器的80端口映射为宿主机的8080端口。
这样创建的pod类似于docker端口映射：docker run -p 80 nginx 只写一个端口80的话表示容器端口，docker run -p 80:80 nginx 80:80 表示宿主机端口:容器端口。
yaml
[root@etcd2 minsvcbug]# vim podport.yaml
[root@etcd2 minsvcbug]# cat podport.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
resources: {}
volumeMounts:
- name: v1
mountPath: /data
ports:
- name: http
containerPort: 80
hostPort: 8080
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}创建pod的时候发现tom用户没有pods的patch权限，我们去授权一下。
shell
[root@etcd2 minsvcbug]# kubectl apply -f podport.yaml -n minsvcbug
for: "podport.yaml": pods "podtest" is forbidden: User "tom" cannot patch resource "pods" in API group "" in the namespace "minsvcbug"修改Role配置文件，添加patch权限。
yaml
root@k8scludes1:~/minsvcbug# vim role1.yaml
root@k8scludes1:~/minsvcbug# cat role1.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
creationTimestamp: null
name: role1
rules:
- apiGroups:
- ""
resources:
- pods
- pods/exec
- pods/log
- services
- secrets
verbs:
- get
- list
- watch
- create
- delete
#添加patch权限
- patch应用角色。
shell
root@k8scludes1:~/minsvcbug# kubectl apply -f role1.yaml
role.rbac.authorization.k8s.io/role1 configured客户端继续创建pod。
创建pod失败，Invalid value: 8080: Host port 8080 is not allowed to be used. Allowed ports: [] 表示PSP规则规定的Host port为空，不能使用hostport端口，我们需要修改PSP规则。
shell
[root@etcd2 minsvcbug]# kubectl get pod -n minsvcbug
No resources found in minsvcbug namespace.
[root@etcd2 minsvcbug]# kubectl apply -f podport.yaml -n minsvcbug
Error from server (Forbidden): error when creating "podport.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].hostPort: Invalid value: 8080: Host port 8080 is not allowed to be used. Allowed ports: []]修改PSP规则，hostPorts:- min: 1000 max: 8090表示hostport的端口范围为1000-8090。
yaml
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
#hostNetwork: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
#rule: MustRunAs
#ranges:
#- min: 1000
# max: 2000
fsGroup:
rule: RunAsAny
volumes:
- '*'
#- 'emptyDir'
#- 'projected'
#allowedHostPaths:
#- pathPrefix: "/tmp"
#指定hostport的端口范围为1000-8090
hostPorts:
- min: 1000
max: 8090应用PSP规则。
shell
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod configured
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false *
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir现在客户端创建pod就成功了。
shell
[root@etcd2 minsvcbug]# kubectl apply -f podport.yaml -n minsvcbug
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -o wide -n minsvcbug
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 53s 10.244.218.150 k8scludes2 <none> <none>访问nginx的方法为：curl k8scludes2的ip:8080，nginx访问成功。
html
[root@etcd2 minsvcbug]# curl 192.168.110.129:8080
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>删除pod。
shell
[root@etcd2 minsvcbug]# kubectl delete pod podtest -n minsvcbug
pod "podtest" deleted
[root@etcd2 minsvcbug]# kubectl get pod -o wide -n minsvcbug
No resources found in minsvcbug namespace.删除rolebinding。
shell
root@k8scludes1:~# cd minsvcbug/
root@k8scludes1:~/minsvcbug# kubectl get rolebinding
NAME ROLE AGE
rolebinding-clusterrole-mount-psp ClusterRole/clusterrole--psp-mount-rule 3d21h
rolebindingtotom Role/role1 5d3h
root@k8scludes1:~/minsvcbug# kubectl delete rolebinding rolebinding-clusterrole-mount-psp
rolebinding.rbac.authorization.k8s.io "rolebinding-clusterrole-mount-psp" deleted
root@k8scludes1:~/minsvcbug# kubectl get rolebinding
NAME ROLE AGE
rolebindingtotom Role/role1 5d3h
root@k8scludes1:~/minsvcbug# kubectl delete rolebinding rolebindingtotom
rolebinding.rbac.authorization.k8s.io "rolebindingtotom" deleted
root@k8scludes1:~/minsvcbug# kubectl get rolebinding
No resources found in minsvcbug namespace.删除role。
shell
root@k8scludes1:~/minsvcbug# kubectl get role
NAME CREATED AT
role1 2022-05-20T03:42:48Z
root@k8scludes1:~/minsvcbug# kubectl delete role role1
role.rbac.authorization.k8s.io "role1" deleted
root@k8scludes1:~/minsvcbug# kubectl get role
No resources found in minsvcbug namespace.删除PSP规则。
shell
root@k8scludes1:~/minsvcbug# kubectl get psp
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false *
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
#删除psp-mount-pod规则
root@k8scludes1:~/minsvcbug# kubectl delete psp psp-mount-pod
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy "psp-mount-pod" deleted十二.PSP规则实战1
我们现在要解决的问题是：有三个命名空间ns1,ns2,ns3，tom用户只有在ns1命名空间里可以创建特权用户pod，其他命名空间都不能创建特权用户pod，需要怎么做？
创建3个命名空间。
shell
root@k8scludes1:~/minsvcbug# kubectl create ns ns1
namespace/ns1 created
root@k8scludes1:~/minsvcbug# kubectl create ns ns2
namespace/ns2 created
root@k8scludes1:~/minsvcbug# kubectl create ns ns3
namespace/ns3 created创建一个普通的RBAC权限，让客户端tom用户可以在所有的命名空间里创建和删除pod,使用clusterrole和clusterrolebinding。
编辑ClusterRole配置文件，clusterole1具有对pod的查询/创建/删除权限，因为需要看日志和进入到pod里，添加了pods/exec和pods/log资源。
yaml
root@k8scludes1:~/minsvcbug# vim clusterrole1.yaml
root@k8scludes1:~/minsvcbug# cat clusterrole1.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
creationTimestamp: null
name: clusterole1
rules:
- apiGroups:
- ""
resources:
- pods
- pods/exec
- pods/log
verbs:
- get
- list
- watch
- create
- delete
- patch应用ClusterRole。
shell
root@k8scludes1:~/minsvcbug# kubectl apply -f clusterrole1.yaml
clusterrole.rbac.authorization.k8s.io/clusterole1 created
root@k8scludes1:~/minsvcbug# kubectl get clusterrole | grep clusterole1
clusterole1 2022-05-25T07:31:28Z查看clusterole1权限信息。
shell
root@k8scludes1:~/minsvcbug# kubectl describe clusterrole clusterole1
Name: clusterole1
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
pods/exec [] [] [get list watch create delete patch]
pods/log [] [] [get list watch create delete patch]
pods [] [] [get list watch create delete patch]把clusterole1绑定给tom用户，clusterole1和clusterrolebinding1的目的是让tom用户可以在所有命名空间里创建和删除pod。
现在客户端的tom用户在所有命名空间都可以创建和删除pod了。
shell
root@k8scludes1:~/minsvcbug# kubectl create clusterrolebinding clusterrolebinding1 --clusterrole=clusterole1 --user=tom
clusterrolebinding.rbac.authorization.k8s.io/clusterrolebinding1 created
root@k8scludes1:~/minsvcbug# kubectl get clusterrolebinding | grep clusterrolebinding1
clusterrolebinding1 ClusterRole/clusterole1 21s创建两条psp规则，一条PSP规则禁止创建特权用户pod，另外一条PSP规则允许创建特权用户pod。
编辑psp-allowpri-pod规则，psp-allowpri-pod规则允许创建特权用户pod，privileged: true。
yaml
root@k8scludes1:~/minsvcbug# vim psp-allowpri-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-allowpri-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-allowpri-pod
spec:
#allow privileged pods!
privileged: true
#hostNetwork: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
#rule: MustRunAs
#ranges:
#- min: 1000
# max: 2000
fsGroup:
rule: RunAsAny
volumes:
- '*'
#- 'emptyDir'
#- 'projected'
#allowedHostPaths:
#- pathPrefix: "/tmp"
#hostPorts:
#- min: 1000
# max: 8090编辑psp-denypri-pod规则，psp-allowpri-pod规则禁止创建特权用户pod，privileged: false。
yaml
root@k8scludes1:~/minsvcbug# vim psp-denypri-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-denypri-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-denypri-pod
spec:
#allow privileged pods!
privileged: false
#hostNetwork: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
#rule: MustRunAs
#ranges:
#- min: 1000
# max: 2000
fsGroup:
rule: RunAsAny
volumes:
- '*'
#- 'emptyDir'
#- 'projected'
#allowedHostPaths:
#- pathPrefix: "/tmp"
#hostPorts:
#- min: 1000
# max: 8090创建PSP规则。
shell
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-allowpri-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-allowpri-pod created
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-denypri-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-denypri-pod created
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-allowpri-pod true RunAsAny RunAsAny RunAsAny RunAsAny false *
psp-denypri-pod false RunAsAny RunAsAny RunAsAny RunAsAny false *
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir为了达到tom用户在所有的namespace里都不能创建特权用户pod的效果：需要通过clusterrole和clusterrolebinding把psp-denypri-pod规则授权给tom用户。
现在的clusterrole，clusterrolebinding如下：
shell
root@k8scludes1:~/minsvcbug# kubectl get clusterrole | grep cluste
cert-manager-controller-clusterissuers 2022-04-24T17:07:47Z
cluster-admin 2022-04-16T14:57:55Z
clusterole1 2022-05-25T07:31:28Z
clusterrole--psp-mount-rule 2022-05-21T09:42:10Z
system:controller:clusterrole-aggregation-controller 2022-04-16T14:57:55Z
root@k8scludes1:~/minsvcbug# kubectl get clusterrolebinding | grep cluster
cert-manager-controller-clusterissuers ClusterRole/cert-manager-controller-clusterissuers 30d
cluster-admin ClusterRole/cluster-admin 38d
clusterrolebinding1 ClusterRole/clusterole1 30m
system:controller:clusterrole-aggregation-controller ClusterRole/system:controller:clusterrole-aggregation-controller 38d创建clusterrole，角色clusterrole--psp-denypri-rule对名为psp-denypri-pod的psp规则有use权限。
shell
root@k8scludes1:~/minsvcbug# kubectl create clusterrole clusterrole--psp-denypri-rule --verb=use --resource=psp --resource-name=psp-denypri-pod
clusterrole.rbac.authorization.k8s.io/clusterrole--psp-denypri-rule created查看clusterrole--psp-denypri-rule的描述信息。
shell
root@k8scludes1:~/minsvcbug# kubectl describe clusterrole clusterrole--psp-denypri-rule
Name: clusterrole--psp-denypri-rule
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
podsecuritypolicies.policy [] [psp-denypri-pod] [use]把角色clusterrole--psp-denypri-rule绑定给tom用户，clusterrole--psp-denypri-rule和clusterrolebinding-clusterrole-denypri-psp的目的是让tom具有psp-denypri-pod规则的use权限，这样tom在所有的命名空间就不能创建特权用户pod了。
shell
root@k8scludes1:~/minsvcbug# kubectl create clusterrolebinding clusterrolebinding-clusterrole-denypri-psp --clusterrole=clusterrole--psp-denypri-rule --user=tom
clusterrolebinding.rbac.authorization.k8s.io/clusterrolebinding-clusterrole-denypri-psp created在客户端进行验证，首先测试在多个namespace创建和删除pod。
编辑pod配置文件，这是非特权用户pod。
yaml
[root@etcd2 minsvcbug]# vim podnotpri.yaml
[root@etcd2 minsvcbug]# cat podnotpri.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
resources: {}
volumeMounts:
- name: v1
mountPath: /data
#ports:
#- name: http
# containerPort: 80
# hostPort: 8080
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}tom用户在ns1，ns2，ns3都能创建非特权pod。
shell
[root@etcd2 minsvcbug]# kubectl apply -f podnotpri.yaml -n ns1
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -n ns1
NAME READY STATUS RESTARTS AGE
podtest 1/1 Running 0 12s
[root@etcd2 minsvcbug]# kubectl apply -f podnotpri.yaml -n ns2
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -n ns2
NAME READY STATUS RESTARTS AGE
podtest 1/1 Running 0 7s
[root@etcd2 minsvcbug]# kubectl apply -f podnotpri.yaml -n ns3
pod/podtest createdtom用户在ns1，ns2，ns3都能删除pod。
shell
[root@etcd2 minsvcbug]# kubectl delete -f podnotpri.yaml -n ns1
pod "podtest" deleted
[root@etcd2 minsvcbug]# kubectl delete -f podnotpri.yaml -n ns2
pod "podtest" deleted
#这样tom用户就可以在ns1，ns2，ns3创建和删除pod了
[root@etcd2 minsvcbug]# kubectl delete -f podnotpri.yaml -n ns3
pod "podtest" deleted编辑pod配置文件，securityContext:privileged: true 表示创建特权用户pod。
yaml
[root@etcd2 minsvcbug]# vim podpri.yaml
[root@etcd2 minsvcbug]# cat podpri.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
#securityContext:privileged: true 表示创建特权用户pod
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
#ports:
#- name: http
# containerPort: 80
# hostPort: 8080
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}可以发现，tom用户可以在ns1，ns2，ns3命名空间创建pod，但是不能创建特权pod。
shell
[root@etcd2 minsvcbug]# kubectl apply -f podpri.yaml -n ns1
Error from server (Forbidden): error when creating "podpri.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]
[root@etcd2 minsvcbug]# kubectl apply -f podpri.yaml -n ns2
Error from server (Forbidden): error when creating "podpri.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]
[root@etcd2 minsvcbug]# kubectl apply -f podpri.yaml -n ns3
Error from server (Forbidden): error when creating "podpri.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]创建clusterrole，角色clusterrole--psp-allowpri-rule对名为psp-allowpri-pod的psp规则有use权限。
shell
root@k8scludes1:~/minsvcbug# kubectl create clusterrole clusterrole--psp-allowpri-rule --verb=use --resource=psp --resource-name=psp-allowpri-pod
clusterrole.rbac.authorization.k8s.io/clusterrole--psp-allowpri-rule created把角色clusterrole--psp-allowpri-rule绑定给tom用户，这样tom用户既可以访问psp-allowpri-pod规则，也可以访问psp-denypri-pod规则。
shell
root@k8scludes1:~/minsvcbug# kubectl create rolebinding rolebinding-clusterrole-allowpri-psp --clusterrole=clusterrole--psp-allowpri-rule --user=tom
rolebinding.rbac.authorization.k8s.io/rolebinding-clusterrole-allowpri-psp created删除rolebinding-clusterrole-allowpri-psp，因为没有指定是哪个命名空间，不指定的话默认是当前命名空间。
shell
root@k8scludes1:~/minsvcbug# kubectl delete rolebinding rolebinding-clusterrole-allowpri-psp
rolebinding.rbac.authorization.k8s.io "rolebinding-clusterrole-allowpri-psp" deleted重新进行角色绑定，这样tom用户在ns1命名空间既可以访问psp-allowpri-pod规则，也可以访问psp-denypri-pod规则。
注意：对于PSP规则，拒绝和允许同时出现的话，允许优先级比较高。
shell
root@k8scludes1:~/minsvcbug# kubectl create rolebinding rolebinding-clusterrole-allowpri-psp --clusterrole=clusterrole--psp-allowpri-rule --user=tom -n ns1
rolebinding.rbac.authorization.k8s.io/rolebinding-clusterrole-allowpri-psp created下面在客户端进行测试。
编辑pod配置文件，下面的配置文件是为了创建特权用户pod。
yaml
[root@etcd2 minsvcbug]# cat podpri.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
#创建特权pod
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
#ports:
#- name: http
# containerPort: 80
# hostPort: 8080
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}在ns1命名空间创建特权pod成功。
shell
[root@etcd2 minsvcbug]# kubectl apply -f podpri.yaml -n ns1
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -o wide -n ns1
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 12s 10.244.218.148 k8scludes2 <none> <none>在ns2，ns3命名空间创建特权pod失败。
shell
[root@etcd2 minsvcbug]# kubectl apply -f podpri.yaml -n ns2
Error from server (Forbidden): error when creating "podpri.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]
[root@etcd2 minsvcbug]# kubectl apply -f podpri.yaml -n ns3
Error from server (Forbidden): error when creating "podpri.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]这样就实现了tom用户可以在ns1命名空间创建特权用户pod，在其他命名空间都不可以创建特权pod的需求。
清理一下pod。
shell
[root@etcd2 minsvcbug]# kubectl delete -f podpri.yaml -n ns1
pod "podtest" deleted
[root@etcd2 minsvcbug]# kubectl get pod -o wide -n ns1
No resources found in ns1 namespace.
[root@etcd2 minsvcbug]# kubectl get pod -o wide -n ns2
No resources found in ns2 namespace.
[root@etcd2 minsvcbug]# kubectl get pod -o wide -n ns3
No resources found in ns3 namespace.十三.PSP规则实战2:创建deployment
现在没有deploy，使用hub.c.163.com/library/nginx:latest镜像创建一个deploy，先生成yaml文件。关于deploy的详细操作，请查看博客《Kubernetes(k8s)控制器(一)：deployment》。
shell
root@k8scludes1:~/minsvcbug# kubectl get deploy
No resources found in minsvcbug namespace.
root@k8scludes1:~/minsvcbug# kubectl create deployment nginxdeploy --image=hub.c.163.com/library/nginx:latest --dry-run=client -o yaml >nginxdeploy.yaml生成的deploy配置文件如下：
yaml
root@k8scludes1:~/minsvcbug# vim nginxdeploy.yaml
root@k8scludes1:~/minsvcbug# cat nginxdeploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
#这个labels是deployment的标签
labels:
app: nginxdeploy
#deployment名字
name: nginxdeploy
spec:
#replicas: 1 表示副本数为1，只生成一个pod
replicas: 1
#selector标签选择器
selector:
matchLabels:
app: nginxdeploy
strategy: {}
#template下面是pod的模板
template:
metadata:
creationTimestamp: null
#这个labels表示pod标签
labels:
app: nginxdeploy
spec:
#当需要关闭容器时，立即杀死容器而不等待默认的30秒优雅停机时长。
terminationGracePeriodSeconds: 0
containers:
- image: hub.c.163.com/library/nginx:latest
#imagePullPolicy: IfNotPresent：表示如果本地已经存在该镜像，则不重新下载；否则从远程 Docker Hub 下载该镜像
imagePullPolicy: IfNotPresent
name: nginx
resources: {}
status: {}创建deploy。
shell
root@k8scludes1:~/minsvcbug# kubectl apply -f nginxdeploy.yaml
deployment.apps/nginxdeploy created发现deploy不正常。
shell
root@k8scludes1:~/minsvcbug# kubectl get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginxdeploy 0/1 0 0 18s nginx hub.c.163.com/library/nginx:latest app=nginxdeploypod也没有生成。
shell
root@k8scludes1:~/minsvcbug# kubectl get pod -o wide
No resources found in minsvcbug namespace.注意 ：当我们创建一个deploy，比如nginxdeploy，则系统会为我们创建一个replicaset，名字为nginxdeploy-XXX，我们在deploy里指定的副本，本质上是由replicaset创建出来的，pod名字为nginxdeploy-XXX-yyyy。
可以发现，deployment和replicaset都没有ready。
shell
root@k8scludes1:~/minsvcbug# kubectl get all -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
deployment.apps/nginxdeploy 0/1 0 0 7m17s nginx hub.c.163.com/library/nginx:latest app=nginxdeploy
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
replicaset.apps/nginxdeploy-5d458fcbfc 1 0 0 7m17s nginx hub.c.163.com/library/nginx:latest app=nginxdeploy,pod-template-hash=5d458fcbfc在kube-system命名空间里有很多个给控制器使用的sa。
shell
root@k8scludes1:~/minsvcbug# kubectl get sa -n kube-system
NAME SECRETS AGE
attachdetach-controller 1 38d
bootstrap-signer 1 38d
calico-kube-controllers 1 38d
calico-node 1 38d
certificate-controller 1 38d
......
persistent-volume-binder 1 38d
pod-garbage-collector 1 38d
pv-protection-controller 1 38d
pvc-protection-controller 1 38d
replicaset-controller 1 38d
replication-controller 1 38d
resourcequota-controller 1 38d
root-ca-cert-publisher 1 38d
service-account-controller 1 38d
service-controller 1 38d
statefulset-controller 1 38d
token-cleaner 1 38d
ttl-after-finished-controller 1 38d
ttl-controller 1 38d只查看和controller有关的SA。
shell
root@k8scludes1:~/minsvcbug# kubectl get sa -n kube-system | egrep '*controller'
attachdetach-controller 1 38d
calico-kube-controllers 1 38d
certificate-controller 1 38d
clusterrole-aggregation-controller 1 38d
cronjob-controller 1 38d
daemon-set-controller 1 38d
deployment-controller 1 38d
disruption-controller 1 38d
endpoint-controller 1 38d
endpointslice-controller 1 38d
endpointslicemirroring-controller 1 38d
ephemeral-volume-controller 1 38d
expand-controller 1 38d
job-controller 1 38d
namespace-controller 1 38d
node-controller 1 38d
pv-protection-controller 1 38d
pvc-protection-controller 1 38d
replicaset-controller 1 38d
replication-controller 1 38d
resourcequota-controller 1 38d
service-account-controller 1 38d
service-controller 1 38d
statefulset-controller 1 38d
ttl-after-finished-controller 1 38d
ttl-controller 1 38d控制器创建pod的过程：在kube-system里有很多个给控制器使用的sa,启用PSP之后，现在控制器的SA都访问不了PSP规则，就创建不了pod,只有控制器的sa能访问PSP规则，才能正常创建pod。
现在需要给控制器SA进行授权，使控制器能访问PSP规则。
进行角色绑定，表示kube-system命名空间里的所有serviceaccounts都可以访问psp-allowpri-pod规则。
shell
root@k8scludes1:~/minsvcbug# kubectl get clusterrole | grep cluster
cert-manager-controller-clusterissuers 2022-04-24T17:07:47Z
cluster-admin 2022-04-16T14:57:55Z
clusterole1 2022-05-25T07:31:28Z
clusterrole--psp-allowpri-rule 2022-05-25T08:57:00Z
clusterrole--psp-denypri-rule 2022-05-25T08:09:01Z
system:controller:clusterrole-aggregation-controller 2022-04-16T14:57:55Z
root@k8scludes1:~/minsvcbug# kubectl create rolebinding rolebind-allowpri-psp-kube-system --clusterrole=clusterrole--psp-allowpri-rule --group=system:serviceaccounts -n kube-system --dry-run=client -o yaml > rolebind-allowpri-kube-system.yaml
root@k8scludes1:~/minsvcbug# kubectl apply -f rolebind-allowpri-kube-system.yaml
rolebinding.rbac.authorization.k8s.io/rolebind-allowpri-psp-kube-system created删除创建失败的nginxdeploy。
shell
root@k8scludes1:~/minsvcbug# kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
nginxdeploy 0/1 0 0 3h13m
root@k8scludes1:~/minsvcbug# kubectl delete deploy nginxdeploy
deployment.apps "nginxdeploy" deleted
root@k8scludes1:~/minsvcbug# kubectl get deployments
No resources found in minsvcbug namespace.再次创建deploy。
shell
root@k8scludes1:~/minsvcbug# kubectl apply -f nginxdeploy.yaml
deployment.apps/nginxdeploy createddeploy还是没有创建成功。
shell
root@k8scludes1:~/minsvcbug# kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
nginxdeploy 0/1 0 0 8s
root@k8scludes1:~/minsvcbug# kubectl get pod
No resources found in minsvcbug namespace.查看nginxdeploy描述信息：replicaset-controller Error creating: pods "nginxdeploy-5d458fcbfc-" is forbidden: PodSecurityPolicy: unable to admit pod: []，可以发现权限还是不够。
shell
root@k8scludes1:~/minsvcbug# kubectl describe deployment nginxdeploy
Name: nginxdeploy
Namespace: minsvcbug
......
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
ReplicaFailure True FailedCreate
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedCreate 34s (x14 over 75s) replicaset-controller Error creating: pods "nginxdeploy-5d458fcbfc-" is forbidden: PodSecurityPolicy: unable to admit pod: []kubectl get ev查看事件。
shell
root@k8scludes1:~/minsvcbug# kubectl get ev
LAST SEEN TYPE REASON OBJECT MESSAGE
11m Warning FailedCreate replicaset/nginxdeploy-5d458fcbfc Error creating: pods "nginxdeploy-5d458fcbfc-" is forbidden: PodSecurityPolicy: unable to admit pod: []
101s Warning FailedCreate replicaset/nginxdeploy-5d458fcbfc Error creating: pods "nginxdeploy-5d458fcbfc-" is forbidden: PodSecurityPolicy: unable to admit pod: []
4m25s Normal ScalingReplicaSet deployment/nginxdeploy Scaled up replica set nginxdeploy-5d458fcbfc to 1删除nginxdeploy。
shell
root@k8scludes1:~/minsvcbug# kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
nginxdeploy 0/1 0 0 5m18s
root@k8scludes1:~/minsvcbug# kubectl delete deploy nginxdeploy
deployment.apps "nginxdeploy" deleted
root@k8scludes1:~/minsvcbug# kubectl get deployments
No resources found in minsvcbug namespace.
root@k8scludes1:~/minsvcbug# kubectl get all
No resources found in minsvcbug namespace.编辑ClusterRoleBinding配置文件，把RoleBinding变为ClusterRoleBinding，使其全局生效，刚才的角色绑定rolebind-allowpri-psp-kube-system是错误的。
yaml
root@k8scludes1:~/minsvcbug# vim rolebind-allowpri-kube-system.yaml
root@k8scludes1:~/minsvcbug# cat rolebind-allowpri-kube-system.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
creationTimestamp: null
name: clusterrolebind-allowpri-psp-kube-system
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: clusterrole--psp-allowpri-rule
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:serviceaccounts删除rolebind-allowpri-psp-kube-system。
shell
root@k8scludes1:~/minsvcbug# kubectl get rolebinding -n kube-system | grep rolebind-allowpri-psp-kube-system
rolebind-allowpri-psp-kube-system ClusterRole/clusterrole--psp-allowpri-rule 12m
root@k8scludes1:~/minsvcbug# kubectl delete rolebinding rolebind-allowpri-psp-kube-system -n kube-system
rolebinding.rbac.authorization.k8s.io "rolebind-allowpri-psp-kube-system" deleted
root@k8scludes1:~/minsvcbug# kubectl get rolebinding -n kube-system | grep rolebind-allowpri-psp-kube-system创建clusterrolebinding。
shell
root@k8scludes1:~/minsvcbug# kubectl apply -f rolebind-allowpri-kube-system.yaml
clusterrolebinding.rbac.authorization.k8s.io/clusterrolebind-allowpri-psp-kube-system created
root@k8scludes1:~/minsvcbug# kubectl get clusterrolebinding -n kube-system | egrep clusterrolebind-allowpri-psp-kube-system
clusterrolebind-allowpri-psp-kube-system ClusterRole/clusterrole--psp-allowpri-rule 45s创建deployment。
shell
root@k8scludes1:~/minsvcbug# kubectl apply -f nginxdeploy.yaml
deployment.apps/nginxdeploy created现在deploy就创建成功了。
shell
root@k8scludes1:~/minsvcbug# kubectl get all -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/nginxdeploy-5d458fcbfc-rs42b 1/1 Running 0 12s 10.244.218.160 k8scludes2 <none> <none>
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
deployment.apps/nginxdeploy 1/1 1 1 12s nginx hub.c.163.com/library/nginx:latest app=nginxdeploy
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
replicaset.apps/nginxdeploy-5d458fcbfc 1 1 1 12s nginx hub.c.163.com/library/nginx:latest app=nginxdeploy,pod-template-hash=5d458fcbfc
root@k8scludes1:~/minsvcbug# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginxdeploy-5d458fcbfc-rs42b 1/1 Running 0 71sclusterrolebinding全局生效，所有命名空间都能看到。
shell
root@k8scludes1:~/minsvcbug# kubectl get clusterrolebinding | egrep clusterrolebind-allowpri-psp-kube-system
clusterrolebind-allowpri-psp-kube-system ClusterRole/clusterrole--psp-allowpri-rule 4m43s
root@k8scludes1:~/minsvcbug# kubectl get clusterrolebinding -n default | egrep clusterrolebind-allowpri-psp-kube-system
clusterrolebind-allowpri-psp-kube-system ClusterRole/clusterrole--psp-allowpri-rule 4m53s删除nginxdeploy。
shell
root@k8scludes1:~/minsvcbug# kubectl delete deploy nginxdeploy
deployment.apps "nginxdeploy" deleted
root@k8scludes1:~/minsvcbug# kubectl get all
No resources found in minsvcbug namespace.十四.总结
PodSecurityPolicy 是 Kubernetes 集群安全的重要组成部分。通过定义 Pod 的安全策略，管理员可以确保集群中的 Pod 遵循特定的安全标准，从而降低潜在的安全风险。本文介绍了 PSP 的概念、重要性以及如何在 Kubernetes 集群中实施。希望这些信息能够帮助您更好地保护您的 Kubernetes 集群。