docker in docker 在CI中应用解析

docker in docker 简介

docker里嵌套运行docker，本文讲解其在jenkins和gitlab-runner 种的调用流程

一、用于jenkins

容器化部署jenkins时调用docker命令集成CI功能

shell 复制代码

[root@ops-demo~]# docker inspect jenkins  --format="{{json .Mounts}}" 
[{"Type":"bind","Source":"/usr/bin/docker","Destination":"/usr/bin/docker","Mode":"","RW":true,"Propagation":"rprivate"},{"Type":"volume","Name":"jenkins","Source":"/var/lib/docker/volumes/jenkins/_data","Destination":"/var/jenkins_home","Driver":"local","Mode":"z","RW":true,"Propagation":""},{"Type":"bind","Source":"/var/run/docker.sock","Destination":"/var/run/docker.sock","Mode":"","RW":true,"Propagation":"rprivate"}]

1.1 调用流程

执行如下命令可以发现jenkins是使用挂载进宿主机的docker.sock 来调用的docker服务，即：在jenkins容器内和宿主机上执行docker命令，效果是一样的，操作的结果，容器内外查看效果一样，要是构建的时候在jenkins容器执行 docker stop jenkins 会发生什么？

shell 复制代码

[root@dotnet ~]# docker images |  wc -l
124
[root@dotnet ~]# docker exec -it jenkins bash
jenkins@855107e4687c:/$ docker images | wc -l
124
jenkins@855107e4687c:/$ 

# 容器内外的镜像数据一样，说明用的是用一个docker服务

二、用于gitlab-runner

gitlab-runner可以用容器方式和非容器方式运行，本文讲docker套娃，于是采用容器方式运行gitlab-runner。

gitlab-runner的executor有多种，对于docker套娃有两种方式，下文讲第二种调用流程。

1、向上文jenkins一样，直接挂载/var/run/docker.sock
2、通过在gitlab-runner里新建一个完整的docker服务，这个完整的服务docker官方有提供一个镜像: docker:dind

2.1 docker套娃调用流程解析

注意看中文注释

gitlab-runner config.toml 配置样例

toml 复制代码

[[runners]]
  url = "https://gitlab.com/"
  token = TOKEN
  executor = "docker" #docker 执行器
  [runners.docker]
    tls_verify = false
    image = "docker:24.0.5" # 用于gitalb-runner容器里提供docker client
    privileged = true
    disable_cache = false
    volumes = ["/cache"]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]

gitlab Project中的 gitlab-ci.yaml 配置样例

yaml 复制代码

default:
  image: docker:24.0.5 #用于提供 docker client
  services:
    - docker:24.0.5-dind # 用于启动 Docker daemon
  before_script:
    - docker info

variables:
  # When using dind service, you must instruct docker to talk with the
  # daemon started inside of the service. The daemon is available with
  # a network connection instead of the default /var/run/docker.sock socket.  # 注意这句提示
  #
  # The 'docker' hostname is the alias of the service container as described at
  # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services
  #
  # If you're using GitLab Runner 12.7 or earlier with the Kubernetes executor and Kubernetes 1.6 or earlier,
  # the variable must be set to tcp://localhost:2375 because of how the
  # Kubernetes executor connects services to the job container
  # DOCKER_HOST: tcp://localhost:2375
  #
  DOCKER_HOST: tcp://docker:2376 #dind启动的Docker daemon所监听的tcp端口
  #
  # This instructs Docker not to start over TLS.
  DOCKER_TLS_CERTDIR: ""

build:
  stage: build
  script:
    - docker build -t my-docker-image .
    - docker run my-docker-image /script/to/run/tests

启动docker:24.0.5看看

[root@mq01 ~]# docker run -it --privileged --name docker-client docker:24.0.5 sh/ # docker pserror during connect: Get "http://docker:2375/v1.24/containers/json": dial tcp: lookup docker on xxx: no such host

可以看到默认请求的host是docker ，但是端口为什么不是2376呢？再看

shell 复制代码

[root@mq01 ~]# docker run -it  --privileged --name docker-client docker:24.0.5 sh
/ # env
HOSTNAME=c3b5946753f1
SHLVL=3
HOME=/root
DIND_COMMIT=d58df1fc6c866447ce2cd129af10e5b507705624
TERM=xterm
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
DOCKER_VERSION=24.0.5
DOCKER_TLS_CERTDIR=/certs
DOCKER_HOST=tcp://docker:2375 # 这里通过环境变量设置的，上文gitlab-ci.yaml中自定义为dind默认监听的端口2376
DOCKER_BUILDX_VERSION=0.11.2
DOCKER_COMPOSE_VERSION=2.21.0
PWD=/
/ #

启动dind验证下监听的是不是2376端口？

time="2024-05-29T10:16:09.048087250Z" level=info msg="API listen on [::]:2376"

shell 复制代码

[root@mq01 ~]# docker run --privileged --name dind docker:dind 
...
time="2024-05-29T10:16:08.938258364Z" level=info msg="Docker daemon" commit=8e96db1 containerd-snapshotter=false storage-driver=overlay2 version=26.1.3
time="2024-05-29T10:16:08.939055975Z" level=info msg="Daemon has completed initialization"
time="2024-05-29T10:16:09.047817847Z" level=info msg="API listen on /var/run/docker.sock"
time="2024-05-29T10:16:09.048087250Z" level=info msg="API listen on [::]:2376"

2.2 调用流程

经过以上分析，可以捋出gitlab-runer docker套娃的工作流程

1、宿主机docker启动gitlab-runner容器；

2、gitlab-runner调用 docker:24.0.5提供的docker client执行各类docker命令，包括启动docker:dind容器；

3、然后docker client调用的Docker daemon是docker:dind提供的，因此实际的执行环境在docker:dind中。

2.3 gitlab-runner的工作流程

给出一张gitlab-runnner的工作流程图方便理解

希望对您有用！关注锅总，可及时获得更多花里胡哨的运维实用操作！