背景
最近在学习envoy,跟着官方文档做实验,遇到了一些问题,在此记录
1. docker 版本问题
envoy实验使用 Dockerfile 构建镜像时需要使用docker buildx,启动服务时需要使用 docker compose,而系统安装的docker没有这两个组件。最好卸载现有docker重新安装,步骤如下:
bash
sudo apt update
sudo apt install -y ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
https://download.docker.com/linux/ubuntu jammy stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
然后安装docker和需要的插件
bash
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
安装后
bash
root@k8s-master:~/examples# docker --version
Docker version 28.5.0, build 887030f
docker buildx version
root@k8s-master:~/examples# docker buildx version
docker compose version
github.com/docker/buildx v0.29.1 a32761aeb3debd39be1eca514af3693af0db334b
root@k8s-master:~/examples# docker compose version
Docker Compose version v2.40.0
2. 镜像拉取问题
本次实验docker-compose文件如下:
yaml
cat docker-compose.yaml
services:
proxy:
build:
context: .
dockerfile: ../shared/envoy/Dockerfile
depends_on:
- service1
- service2
ports:
- "${PORT_PROXY:-10000}:10000"
- "${PORT_ADMIN:-19000}:19000"
service1:
image: ${COMPOSE_PROJECT_NAME}-echo
build:
context: ../shared/echo
environment:
ECHO_HOSTNAME: service1
service2:
image: ${COMPOSE_PROJECT_NAME}-echo
build:
context: ../shared/echo
environment:
ECHO_HOSTNAME: service2
go-control-plane:
build:
context: .
dockerfile: ../shared/golang/Dockerfile
target: golang-control-plane
command: /usr/local/bin/example
healthcheck:
test: nc -zv localhost 18000
需要编译三个镜像,那就需要从docker国外仓库拉取镜像,但由于国外镜像站无法访问,拉取会报错。
解决办法1:使用镜像代理
bash
vi /etc/docker/daemon.json
{
"registry-mirrors": [
"https://docker.m.daocloud.io"
]
}
但是有些镜像比如本次实验使用的golang:1.24.5-bookworm这个镜像在使用镜像代理拉取时还是会报404,这时候只能科学上网了。
解决办法2:设置代理
export http_proxy=192.168.10.1:7890
export https_proxy=192.168.10.1:7890
3. 镜像编译问题
构建镜像时遇到两个问题:
- 执行 apt-get install 时卡住,执行rustup install nightly时卡住等等
- 执行 RUN --mount=type=cache,target=/var/cache/apt,sharing=locked 卡住等等
这时候就要修改Dockerfile,在合适的地方设置代理,去掉RUN命令中的 --mount 参数,修改后如下
bash
diff --git a/shared/echo/Dockerfile b/shared/echo/Dockerfile
index b88fce8..d4cccab 100644
--- a/shared/echo/Dockerfile
+++ b/shared/echo/Dockerfile
@@ -1,13 +1,15 @@
FROM rust:1.88.0 AS builder
WORKDIR /usr/src/echo
COPY . .
+ENV HTTP_PROXY=192.168.10.1:7890
+ENV HTTPS_PROXY=192.168.10.1:7890
RUN rustup install nightly && rustup default nightly && cargo build --release
FROM debian:bookworm-slim
-RUN --mount=type=tmpfs,target=/var/cache/apt \
- --mount=type=tmpfs,target=/var/lib/apt/lists \
- apt-get -qq update \
+RUN echo 'Acquire::http::Proxy "http://192.168.10.1:7890/";' > /etc/apt/apt.conf.d/01proxy \
+ && echo 'Acquire::https::Proxy "https://192.168.10.1:7890/";' >> /etc/apt/apt.conf.d/01proxy
+RUN apt-get -qq update \
&& apt-get -qq install -y libssl-dev ca-certificates
COPY --from=builder \
/usr/src/echo/target/release/example-echo \
diff --git a/shared/envoy/Dockerfile b/shared/envoy/Dockerfile
index 459841c..f2f95fb 100644
--- a/shared/envoy/Dockerfile
+++ b/shared/envoy/Dockerfile
@@ -7,9 +7,7 @@ ARG ENVOY_CONFIG=envoy.yaml
ENV ENVOY_CONFIG="$ENVOY_CONFIG"
ENV DEBIAN_FRONTEND=noninteractive
RUN echo 'Acquire::Retries "5";' > /etc/apt/apt.conf.d/80-retries
-RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
- --mount=type=cache,target=/var/lib/apt/lists,sharing=locked \
- rm -f /etc/apt/apt.conf.d/docker-clean \
+RUN rm -f /etc/apt/apt.conf.d/docker-clean \
&& echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' | tee /etc/apt/apt.conf.d/keep-cache \
&& apt-get -qq update -y \
&& apt-get -qq install --no-install-recommends -y curl
@@ -28,9 +26,7 @@ HEALTHCHECK \
&& curl -s "localhost:${ENVOY_ADMIN_PORT}/stats?filter=listener_manager.workers_started" | grep 1
FROM envoy-base AS envoy-fault-injection
-RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
- --mount=type=cache,target=/var/lib/apt/lists,sharing=locked \
- apt-get -qq update -y \
+RUN apt-get -qq update -y \
&& apt-get -qq install --no-install-recommends -y tree
COPY enable_delay_fault_injection.sh disable_delay_fault_injection.sh enable_abort_fault_injection.sh disable_abort_fault_injection.sh send_request.sh /
@@ -49,9 +45,7 @@ RUN echo "4a7d17d4724ee890490bcd6cfdedb12a02316a3d33214348d30979abd201f1ca /usr/
FROM envoy-base AS envoy-load-balancing
-RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
- --mount=type=cache,target=/var/lib/apt/lists,sharing=locked \
- apt-get -qq update -y \
+RUN apt-get -qq update -y \
&& apt-get -qq install --no-install-recommends -y python3
COPY ./client.py /client.py
EXPOSE 8001
diff --git a/shared/golang/Dockerfile b/shared/golang/Dockerfile
index b7af9c7..1ffed60 100644
--- a/shared/golang/Dockerfile
+++ b/shared/golang/Dockerfile
@@ -8,6 +8,8 @@ FROM golang:1.24.5-bookworm@sha256:69adc37c19ac6ef724b561b0dc675b27d8c719dfe848d
FROM golang-base AS golang-control-plane-builder
ARG GO_RESOURCE=resource.go
+ENV HTTP_PROXY=192.168.10.1:7890
+ENV HTTPS_PROXY=192.168.10.1:7890
RUN git clone https://github.com/envoyproxy/go-control-plane && cd go-control-plane && git checkout b4adc3bb5fe5288bff01cd452dad418ef98c676e
ADD "$GO_RESOURCE" /go/go-control-plane/internal/example/resource.go
RUN cd go-control-plane && make bin/example
@@ -17,9 +19,9 @@ WORKDIR /go/go-control-plane
FROM os-base AS golang-control-plane
ENV DEBIAN_FRONTEND=noninteractive
RUN echo 'Acquire::Retries "5";' > /etc/apt/apt.conf.d/80-retries
-RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
- --mount=type=cache,target=/var/lib/apt/lists,sharing=locked \
- apt-get -qq update \
+RUN echo 'Acquire::http::Proxy "http://192.168.10.1:7890/";' > /etc/apt/apt.conf.d/01proxy \
+ && echo 'Acquire::https::Proxy "https://192.168.10.1:7890/";' >> /etc/apt/apt.conf.d/01proxy
+RUN apt-get -qq update \
&& apt-get -qq install --no-install-recommends -y netcat-traditional
COPY --from=golang-control-plane-builder /go/go-control-plane/bin/example /usr/local/bin/example
这样服务就可以启动成功了
bash
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
test-go-control-plane-1 test-go-control-plane "/usr/local/bin/exam..." go-control-plane 4 seconds ago Up 4 seconds (health: starting)
test-proxy-1 test-proxy "/docker-entrypoint...." proxy 34 seconds ago Up 33 seconds 0.0.0.0:10000->10000/tcp, [::]:10000->10000/tcp, 0.0.0.0:19000->19000/tcp, [::]:19000->19000/tcp
test-service1-1 test-echo "example-echo -c /et..." service1 34 seconds ago Up 34 seconds 8080/tcp
test-service2-1 test-echo "example-echo -c /et..." service2 34 seconds ago Up 34 seconds 8080/tcp