K8S的mountPath和subPath

1 mountPath

mountPath是容器内部文件系统的挂载点，它定义了容器内部将外部存储卷（如 PersistentVolume、ConfigMap、Secret 等）挂载到哪个路径下。通过 mountPath，容器可以访问这些挂载的数据或配置。

2 subPath

subPath 是 mountPath 下的子路径，它允许容器将挂载的数据卷中的特定文件或目录挂载到容器中的指定路径下。这样可以实现更加精细的文件系统级别的访问控制。

3 mountPath使用场景

比如我需要创建一个nginx deployment，需要将自定义的nginx.conf配置文件独立出来，作为一个configmap来挂载到pod中。

apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-config
data:
  my-nginx.conf: |
    server {
        listen       80;
        server_name  localhost;

        location / {
            root   /usr/share/nginx/html;
            index  index.html index.htm;
        }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   /usr/share/nginx/html;
        }
    }
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        volumeMounts:
        - name: nginx-config-volume
          mountPath: /etc/nginx/conf.d
      volumes:
      - name: nginx-config-volume
        configMap:
          name: nginx-config

部署完成后，你可以进入pod，可以看到如下文件，这就是通过configmap挂载到容器中。

kubectl exec -it nginx-deployment-5b4699b7dd-fh4qc -- /bin/sh
# cd /etc/nginx/conf.d
# ls
my-nginx.conf

4 subPath使用场景

如果我想直接通过configmap定义/etc/nginx/nginx.conf，这时候如果还是只使用mountPath，就会有问题了。

apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-config
data:
  nginx.conf: |
    user  nginx;
    worker_processes  auto;

    error_log  /var/log/nginx/error.log notice;
    pid        /var/run/nginx.pid;


    events {
        worker_connections  1024;
    }


    http {
        include       /etc/nginx/mime.types;
        default_type  application/octet-stream;

        log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                          '$status $body_bytes_sent "$http_referer" '
                          '"$http_user_agent" "$http_x_forwarded_for"';

        access_log  /var/log/nginx/access.log  main;

        sendfile        on;
        #tcp_nopush     on;

        keepalive_timeout  65;

        #gzip  on;

        include /etc/nginx/conf.d/*.conf;
    }

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        volumeMounts:
        - name: nginx-config-volume
          mountPath: /etc/nginx
      volumes:
      - name: nginx-config-volume
        configMap:
          name: nginx-config

创建容器后，服务无法起来，会报错，因为此时容器中的/etc/nginx/目录会被我们挂载的configmap给覆盖，所以原先/etc/nginx/目录下的文件都无法被pod访问，也就报错了。

2024/03/25 06:56:58 [emerg] 1#1: open() "/etc/nginx/mime.types" failed (2: No such file or directory) in /etc/nginx/nginx.conf:14
nginx: [emerg] open() "/etc/nginx/mime.types" failed (2: No such file or directory) in /etc/nginx/nginx.conf:14

那如果我将volumeMount改为如下配置呢，

      - name: nginx
        image: nginx
        volumeMounts:
        - name: nginx-config-volume
          mountPath: /etc/nginx/nginx.conf

此时原来/etc/nginx/目录下的文件都不会受影响，但是依旧会报错，连容器都无法创建，

Error: failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/var/lib/kubelet/pods/6755a0be-2f05-4edb-813b-ece2dcc2e8f1/volumes/kubernetes.io~configmap/nginx-config-volume" to rootfs at "/etc/nginx/nginx.conf": mount /var/lib/kubelet/pods/6755a0be-2f05-4edb-813b-ece2dcc2e8f1/volumes/kubernetes.io~configmap/nginx-config-volume:/etc/nginx/nginx.conf (via /proc/self/fd/6), flags: 0x5001: not a directory: unknown

这是因为此时容器中/etc/nginx/nginx.conf这个路径已经是存在的，镜像中默认的文件，没法作为mountpath使用。

当然，你可以将nginx.conf换成其他名字，比如mynginx.conf，

      - name: nginx
        image: nginx
        volumeMounts:
        - name: nginx-config-volume
          mountPath: /etc/nginx/mynginx.conf

这样就不会报错，但是这样的效果是，容器中会创建一个目录/etc/nginx/mynginx.conf/，这个目录下有个文件nginx.conf，与最开始我们在/etc/nginx/conf.d/定义my-nginx.conf一样，但这并不是我们所要的。

kubectl exec -it nginx-deployment-6bf5f55df8-f452d -- /bin/sh
# cd /etc/nginx/mynginx.conf/
# ls
nginx.conf

这个时候，我们就需要使用subPath了，将volumeMount做如下修改，

      - name: nginx
        image: nginx
        volumeMounts:
        - name: nginx-config-volume
          mountPath: /etc/nginx/nginx.conf
          subPath: nginx.conf

这时服务就按照我们预期启动了，容器内文件如下，而且nginx.conf的内容就是我们configmap中定义的配置，

kubectl exec -it nginx-deployment-bb7d454c6-75bwz -- /bin/sh
# cd /etc/nginx/
# ls
conf.d	fastcgi_params	mime.types  modules  nginx.conf  scgi_params  uwsgi_params

类似的场景，比如需要在/etc/nginx/conf.d/为不同的host定义不同的配置，我们就可以创建多个configmap，配合使用subPath来挂载到同一个目录下，

      - name: nginx
        image: nginx
        volumeMounts:
        - name: nginx-config-volume
          mountPath: /etc/nginx/conf.d/nginx.conf
          subPath: nginx.conf
        - name: my-nginx-config-volume
          mountPath: /etc/nginx/conf.d/my-nginx.conf
          subPath: my-nginx.conf

---

参考文档：

1. https://stackoverflow.com/questions/65399714/what-is-the-difference-between-subpath-and-mountpath-in-kubernetes