K8s Secrets：敏感数据安全存储指南

一、Secret概述

k8s secrets**用于存储和管理一些敏感数据，比如密码，token，密钥等敏感信息。**它把 Pod 想要访问的加密数据存放到 Etcd 中。然后用户就可以通过在 Pod 的容器里挂载 Volume 的方式或者环境变量的方式访问到这些 Secret 里保存的信息了。

Secret 类似于 ConfigMap，但专门用于保存机密数据。

二、Secret 类型

内置类型	用法
Opaque	用户定义的任意数据
kubernetes.io/service-account-tokensymotion	服务账号令牌
kubernetes.io/dockercfg	~/.dockercfg 文件的序列化形式
kubernetes.io/dockerconfigjson	~/.docker/config.json 文件的序列化形式
kubernetes.io/basic-auth	用于基本身份认证的凭据
kubernetes.io/ssh-auth	用于 SSH 身份认证的凭据
kubernetes.io/tls	用于 TLS 客户端或者服务器端的数据
bootstrap.kubernetes.io/token	启动引导令牌数据

kubectl 创建类型

复制代码

[root@k8s-master01 ~]# kubectl create secret dotfile -h
Create a secret using specified subcommand.

Available Commands:
  docker-registry   创建一个给 Docker registry 使用的 secret
  generic           Create a secret from a local file, directory, or literal value
  tls               创建一个 TLS secret

Usage:
  kubectl create secret [flags] [options]

Use "kubectl <command> --help" for more information about a given command.
Use "kubectl options" for a list of global command-line options (applies to all commands).

docker-registry: 连接私有镜像仓库的凭证（据）
generic: 常见 secret, 该类型 secret 与 configmap使用相同
tls: 提供 tls 证书, 在 service mesh 中自动挂载

三、Secret 使用

使用场景:

设置容器的环境变量
向 Pod 提供 SSH 密钥或密码等凭据
允许 kubelet 从私有镜像仓库中拉取镜像

Opaque 类型 Secret 的使用

创建

kubectl create

复制代码

[root@k8s-master01 ~]# kubectl create secret generic dotfile --from-literal=username=admin --from-literal=password=123456

[root@k8s-master01 ~]# kubectl get secret dotfile -o yaml
apiVersion: v1
data:
  password: MTIzNDU2
  username: YWRtaW4=
kind: Secret
metadata:
  creationTimestamp: "2024-01-09T07:45:19Z"
  name: dotfile
  namespace: default
  resourceVersion: "621858"
  uid: ce3a3332-5b97-4af0-8312-ced355786e64
type: Opaque

[root@k8s-master01 ~]# echo -n "YWRtaW4=" | base64 -d
admin

yaml

以 yaml 方式创建需要你提前进行 base64

复制代码

[root@k8s-master01 ~]# echo -n "admin" | base64
YWRtaW4=
[root@k8s-master01 ~]# echo -n "123456" | base64
MTIzNDU2

apiVersion: v1
kind: Secret
metadata:
  name: secret-volume
  namespace: default
type: Opaque
data:
  password: MTIzNDU2
  username: YWRtaW4=
immutable: true
####警告信息
Warning: resource secrets/dotfile is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
#表示存在同名的secret

你可以通过将 Secret 的 immutable 字段设置为 true 创建不可更改的 Secret。

创建

复制代码

$ kubectl create -f dotfile-secret.yaml

挂载

作为环境变量

创建 pod

复制代码

[root@k8s-master01 secret]# cat secret-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pod1
spec:
  containers:
  - image: busybox:1.28
    name: busybox
    command: ["/bin/sh","-c","echo $username && env"]
    env:
    - name: username
      valueFrom:
        secretKeyRef:
          key: username
          name: secret-volume    # secret 名称
####pod不会运行，测试时需要查看日志####          
          
######或者########
[root@k8s-master01 secret]# cat secret-pod.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: pod1
spec:
  containers:
  - image: busybox:1.28
    name: busybox
    command: ["/bin/sh","-c","sleep 3600"]
    env:
    - name: username
      valueFrom:
        secretKeyRef:
          key: username
          name: secret-volume    # secret 名称
####pod会启动，需要进入容器中查看####

获取容器日志

复制代码

# 创建
[root@k8s-master01 secret]# kubectl create -f pod1.yaml

[root@k8s-master01 secret]# kubectl logs pod1
admin
KUBERNETES_PORT=tcp://10.10.0.1:443
KUBERNETES_SERVICE_PORT=443
HOSTNAME=pod1
NGINX_SVC_NODEPORT_PORT_8000_TCP_ADDR=10.10.166.16
SHLVL=1
username=admin
HOME=/root
NGINX_SVC_NODEPORT_PORT_8000_TCP_PORT=8000
SERVICE_BLUE_SERVICE_HOST=10.10.157.201
NGINX_SVC_NODEPORT_PORT_8000_TCP_PROTO=tcp
NGINX_SVC_NODEPORT_SERVICE_HOST=10.10.166.16
  RVICE_BLUE_SERVICE_PORT=80
SERVICE_BLUE_PORT=tcp://10.10.157.201:80
NGINX_SVC_NODEPORT_PORT_8000_TCP=tcp://10.10.166.16:8000
KUBERNETES_PORT_443_TCP_ADDR=10.10.0.1
NGINX_SVC_NODEPORT_PORT=tcp://10.10.166.16:8000
NGINX_SVC_NODEPORT_SERVICE_PORT=8000
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
SERVICE_BLUE_PORT_80_TCP_ADDR=10.10.157.201
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_PORT_443_TCP_PROTO=tcp
SERVICE_BLUE_PORT_80_TCP_PORT=80
SERVICE_BLUE_PORT_80_TCP_PROTO=tcp
KUBERNETES_PORT_443_TCP=tcp://10.10.0.1:443
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_SERVICE_HOST=10.10.0.1
PWD=/
SERVICE_BLUE_PORT_80_TCP=tcp://10.10.157.201:80

作为文件挂载及设置 POSIX 权限

创建 pod

复制代码

apiVersion: v1
kind: Pod
metadata:
  name: pod2
spec:
  volumes:
  - name: sec
    secret:
      secretName: secret-volume
      defaultMode: 0400  # 设置文件权限
  containers:
  - image: busybox:1.28
    name: busybox
    command: ["sleep", "24h"]
    volumeMounts:
    - mountPath: /etc/config
      name: sec
注意: secret 挂载到容器后自动 base64 解码

$ kubectl exec pod2 -- ls -l /etc/config/
total 0
lrwxrwxrwx    1 root     root            15 Jan  9 08:00 password -> ..data/password
lrwxrwxrwx    1 root     root            15 Jan  9 08:00 username -> ..data/username

$ kubectl exec pod2 -- cat /etc/config/username
admin

Secret 绑定 serviceAccount（不用做）

k8s 中 pod 会挂载 serviceAccount

挂载路径: /var/run/secrets/kubernetes.io/serviceaccount

挂载内容该 serviceAccount 的: ca.crt, namespace , token

secret:

复制代码

apiVersion: v1
kind: Secret
metadata:
  name: sa-secret
  annotations:
    kubernetes.io/service-account.name: "tdd"
type: kubernetes.io/service-account-token
data:
  password: MTIzNDU2
  username: YWRtaW4=

我们为 secret 添加了 kubernetes.io/service-account.name字段, 为其指定的 serviceAccount, 创建了 Secret 之后，等待 Kubernetes 在 data 字段中填充 token 主键。

查看 secret

复制代码

$ kubectl get secrets sa-secret -o yaml
apiVersion: v1
data:
  ca.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCVENDQWUyZ0F3SUJBZ0lJWVB2VXI2cG02NFF3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TXpFeU16QXdPVE16TVRkYUZ3MHpNekV5TWpjd09UTTRNVGRhTUJVeApFekFSQmdOVkJBTVRDbXQxWW1WeWJtVjBaWE13Z2dFaU1BMEdDU3FHU0liM0RRRUJBUVVBQTRJQkR3QXdnZ0VLCkFvSUJBUUM0M1JFazZPZ0VFMGVWN1JHbmc1YnRONEdHaE5uUFNjbUZuYy9LKzhlSEZESVlPRkRKblZlZ0tOUmQKblVOSEFUL2h3T3RaWFZUTHhieWRGTVNqd0xSeDVPNTg5bUlVM3M2SlVZZjFvajFCUjFsWlkreXA5eVRGdXFYYQphbzc3WTVPbkVpT3BOQXUxek1WWUFRK0V2bzJtbmY2b0p6WDd3SFc2TkxZd0V0TVA2cklhclNaaU11MTllRnBWCnpmNU5RMGFGUU4vbkJyQUpHQm15eHF1cGhNRFpYOXlyK1c3emY4Q296NkxuTHRCMCtIaEtHYm9kUGVyaWk0bjgKczJlc2tLdmZ6NmdJK0dhR3ROYWtNU3gwbVZXV0MycTBDbndrOVJEOTNhd1JWRHd2VnZRczljR1B0R0I3WVdIcApDOXBLeTdaOEgybjdDZndza2FqZTMzWnhuUzJaQWdNQkFBR2pXVEJYTUE0R0ExVWREd0VCL3dRRUF3SUNwREFQCkJnTlZIUk1CQWY4RUJUQURBUUgvTUIwR0ExVWREZ1FXQkJRU1oxaVlWKzlQNTRiNHB3UU9QVkI0TklXYndEQVYKQmdOVkhSRUVEakFNZ2dwcmRXSmxjbTVsZEdWek1BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQkdNRzMvN3dSZwp0ZkxDMkNHTGtRcUdaQmNtTlBzR0lwbmRqSnpTSzVjMXJMSVUrNjNHN3ZnWEJENHVuZ3B5RDFRVE5VSWp1ZVRXCkMxM1lZb1ZjK09zZkIvWFlVRkFvZ1JiWWtvZjV2TTBUeFVzdXN1VVUvT0Q2NDc0cE5xajlzQStrejRiMUliMzAKOVFmUG5mM2ZieVZyVmh0dlYxQmxsNVMwMmVDa2xaazM1SjAzczFLdytOWHovSjQzTDF6UlJjNytCa3M1UVpvbAp2ZnhJYW1BaXdqWGFUMDBhVVB3WjZ4S215KzcvUkZuRjB4aExlanhKRTBrOUU0YWR3NGhWSkNWMUp2aWdzcEpDCmNQL3BJMC9hSlVKeHVKNWZVeENuRVRVOXRpNi9aTGloeDg4Uk1vdmRFS0R4RDZJYlpKSWkxYVJwajhVeGdnL0UKcm1ONU5YTnN3Sk9XCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
  namespace: ZGVmYXVsdA==
  password: MTIzNDU2
  token: ZXlKaGJHY2lPaUpTVXpJMU5pSXNJbXRwWkNJNkluZHBiR05VZVVwd1pUbEpRbEJpV0ZReGFFeHFaamh5WlROWE4wOHdXRXBtU1Y4eGJtODRWblp0UlVFaWZRLmV5SnBjM01pT2lKcmRXSmxjbTVsZEdWekwzTmxjblpwWTJWaFkyTnZkVzUwSWl3aWEzVmlaWEp1WlhSbGN5NXBieTl6WlhKMmFXTmxZV05qYjNWdWRDOXVZVzFsYzNCaFkyVWlPaUprWldaaGRXeDBJaXdpYTNWaVpYSnVaWFJsY3k1cGJ5OXpaWEoyYVdObFlXTmpiM1Z1ZEM5elpXTnlaWFF1Ym1GdFpTSTZJbk5oTFhObFkzSmxkQ0lzSW10MVltVnlibVYwWlhNdWFXOHZjMlZ5ZG1salpXRmpZMjkxYm5RdmMyVnlkbWxqWlMxaFkyTnZkVzUwTG01aGJXVWlPaUowWkdRaUxDSnJkV0psY201bGRHVnpMbWx2TDNObGNuWnBZMlZoWTJOdmRXNTBMM05sY25acFkyVXRZV05qYjNWdWRDNTFhV1FpT2lKbVpEWTRZelkwTVMwMU1qSXpMVFJsTUdRdFlqRXlaQzFpWkdKbE5ETXlOMlpsT0dZaUxDSnpkV0lpT2lKemVYTjBaVzA2YzJWeWRtbGpaV0ZqWTI5MWJuUTZaR1ZtWVhWc2REcDBaR1FpZlEuY09UZXoyM2w3VFl2TjFFN1FYcW4xVkN4dUxhbEJCN0JOMTVyU05IWFZndk5waEY4ZTd3ZkY4R3VheXFaSmZtdXB3ZE5HVzV2SFVRbE8yQVQtX3pGZkEta0hSbEN6OVZzZUVnTV82S0lrZ3FiZlNKNGw2WDRQOWFwaEdvYUhBaEdJUGdibTVGU1UwTWR6NlhzNW9fSnVvVjZfWkxxN3BZRVlGU0Y3REF1U0VyZHljSjdBZFFidWl6cHhuOE5lb3JhZ3Y5OWNDWjFlQjVJS0lGQW05SjBld1dlZEJBOUJhM0haN2RXQWozNE9uLW9mTWVKOWtQcmhvZWlUb2dtSkk5SE9NYWd4Ynh4YmF5cVpYdW9TQWJBSkNqVkRoUVZxdUJZRnJ5T09lWFowSnlXaWkyeVdVQWVvdVNDSEtRWUFXd2pmU2dwaGtKdzFQUkE4M3pFQWFmc3hn
  username: YWRtaW4=
kind: Secret
metadata:
  annotations:
    kubernetes.io/service-account.name: tdd
    kubernetes.io/service-account.uid: fd68c641-5223-4e0d-b12d-bdbe4327fe8f
  creationTimestamp: "2024-01-09T08:49:30Z"
  name: sa-secret
  namespace: default
  resourceVersion: "630499"
  uid: c41bc253-c2c4-4d2f-a5d9-0a4fe8b0d8da
type: kubernetes.io/service-account-token

可以看到 kubernetes 控制器自动为其填充了 namespace, token, ca

TLS Secret

创建方式

yaml 方式创建

复制代码

apiVersion: v1
kind: Secret
metadata:
  name: secret-tls
type: kubernetes.io/tls
data:
  # 值为 base64 编码，这样会掩盖它们，但不会提供任何有用的机密性级别
  tls.crt: |
    LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUNVakNDQWJzQ0FnMytNQTBHQ1NxR1NJYjNE
    UUVCQlFVQU1JR2JNUXN3Q1FZRFZRUUdFd0pLVURFT01Bd0cKQTFVRUNCTUZWRzlyZVc4eEVEQU9C
    ......    
  # 在这个例子中，密钥数据不是真正的 PEM 编码的私钥
  tls.key: |
    RXhhbXBsZSBkYXRhIGZvciB0aGUgVExTIGNydCBmaWVsZA==

kubectl 创建

复制代码

$ kubectl create secret tls my-tls-secret \
  --cert=path/to/cert/file \
  --key=path/to/key/file

Docker 镜像仓库 Secret

yaml 方式创建

复制代码

apiVersion: v1
kind: Secret
metadata:
  name: secret-dockercfg
type: kubernetes.io/dockercfg
data:
  .dockercfg: |
    eyJhdXRocyI6eyJodHRwczovL2V4YW1wbGUvdjEvIjp7ImF1dGgiOiJvcGVuc2VzYW1lIn19fQo=

kubectl 方式创建

复制代码

$ kubectl create secret docker-registry secret-tiger-docker \
  --docker-email=tiger@acme.example \
  --docker-username=tiger \
  --docker-password=pass1234 \
  --docker-server=my-registry.example:5000

$ kubectl get secret secret-tiger-docker -o jsonpath='{.data.*}' | base64 -d

输出等价于以下 JSON 文档（这也是一个有效的 Docker 配置文件）：

复制代码

{
  "auths": {
    "my-registry.example:5000": {
      "username": "tiger",
      "password": "pass1234",
      "email": "tiger@acme.example",
      "auth": "dGlnZXI6cGFzczEyMzQ="
    }
  }
}

ssh 类型 secret

通过文件创建

复制代码

$ kubectl create secret generic ssh-key-secret --from-file=ssh-privatekey=/path/to/.ssh/id_rsa --from-file=ssh-publickey=/path/to/.ssh/id_rsa.pub

pod 挂载 ssh secret

复制代码

apiVersion: v1
kind: Pod
metadata:
  name: secret-test-pod
  labels:
    name: secret-test
spec:
  volumes:
  - name: secret-volume
    secret:
      secretName: ssh-key-secret
  containers:
  - name: ssh-test-container
    image: mySshImage
    volumeMounts:
    - name: secret-volume
      readOnly: true
      mountPath: "/etc/secret-volume"

容器命令执行时，秘钥的数据可以在下面的位置访问到：

复制代码

/etc/secret-volume/ssh-publickey
/etc/secret-volume/ssh-privatekey

容器就可以随便使用 Secret 数据来建立 SSH 连接。