flink operator 拉取阿里云私有镜像（其他私有类似）

创建 k8s secret

kubectl --namespace flink create secret docker-registry aliyun-docker-registry --docker-server=registry.cn-shenzhen.aliyuncs.com --docker-username=ops_acr@1060896234 --docker-password=passwd --docker-email=DOCKER_EMAIL

注意命名空间指定你使用的 我的命名空间是flink

secret type = docker-registry 这个是固定值不需要动

aliyun-docker-registry 为你的secret 取的固定名称

--docker-server 就是你私库地址

--docker-username 登录名称

--docker-password 登录密码

查看

kubectl get secret aliyun-docker-registry -n flink
#后面的命名空间不要忘记 要不然找不到
kubectl  --namespace flink get secret aliyun-docker-registry -o yaml
# 以yaml 的方式查看配置文件

flink yaml

apiVersion: flink.apache.org/v1beta1
kind: FlinkDeployment
metadata:
  name: autoscaling-example
  namespace: flink
spec:
  imagePullPolicy: IfNotPresent
  flinkVersion: v1_18
  flinkConfiguration:
    job.autoscaler.enabled: "true"
    job.autoscaler.stabilization.interval: "1m"
    job.autoscaler.metrics.window: "3m"
    pipeline.max-parallelism: "24"
    taskmanager.numberOfTaskSlots: "2"
    state.savepoints.dir: hdfs:///flink/savepoints
    state.checkpoints.dir: hdfs:///flink/checkpoints
    high-availability.type: kubernetes
    high-availability.storageDir: hdfs:///flink/ha
    execution.checkpointing.interval: "1m"
    jobmanager.scheduler: adaptive
    job.autoscaler.target.utilization.boundary: "0.1"
  serviceAccount: flink
  podTemplate:
    apiVersion: v1
    kind: Pod
    metadata:
      name: pod-template
    spec:
      containers:
        - name: flink-main-container
          imagePullPolicy: IfNotPresent
          image: registry.cn-hangzhou.aliyuncs.com/liguo/autoscaling-example:17.0
      imagePullSecrets:
        - name: aliyun-docker-registry
  jobManager:
    resource:
      memory: "1024m"
      cpu: 1
  taskManager:
    resource:
      memory: "1024m"
      cpu: 1
  job:
    jarURI: local:///opt/flink/usrlib/flink1_18demo-1.0-SNAPSHOT.jar
    parallelism: 1
    upgradeMode: last-state

目前对k8s 刚刚接触1周所有对podTemplate 还是不太了解 目前这个配置时可用的 在pod 中拉取镜像引用 鉴权

提交

kubectl create -f test.yaml -n flink 
#我内部已经指定命名空间了 但是我习惯对外加上

日志持久化

非镜像部署

日志持久化到hdfs