采集华为云 CCI 日志到观测云最佳实践

一、背景与挑战

华为云 CCE 提供了云原生日志采集插件,采集了包含 CCE 集群以及弹性到 CCI 的实例的容器内日志,但对观测云来讲,观测云可以基于 DataKit Operator 以及提供一个 DataKit 的 DaemonSet 部署来实现 CCE 各节点的容器内的日志文件采集,但针对于对于 CCI 的这种 serverless 的容器内日志采集,观测云采集思路包含:

  • 通过观测云的 logforward 的 sidecar 部署来实现日志转发给观测云,这种方式消耗大量的资源,并且要对原有的 CCE 的 Deployment 进行改造注入。
  • 使用 lambda 函数将 LTS 采集的 OBS 的日志上报到观测云,因 CCE 的同一 Deployment 弹性到 CCI,这种方式基于 OBS 区分不出哪些是 CCI 的日志,哪些是 CCE 的日志。
  • 华为云 CCE 云原生日志采集插件中包含了 Otel Collector 组件,通过改造 Otel Collector 的 exporter 配置实现 CCI 日志的导出,这种方式减少了日志接入的成本,避免了资源额外消耗的成本,即本篇重点阐述的最佳实践。

二、前置条件

  • DataKit:观测云的采集组件,负责 CCE 日志采集与接收 Otel Collector 的 CCI 日志收集导出。
  • 观测云:统一日志检索、查询分析、仪表盘展示、智能告警等。
  • 云原生日志采集插件:负责 CCE 日志和 CCI 日志的采集,插件版本要求 1.5.1 版本以上,插件说明如下。
  • 业务场景环境:华为 CCE 调度到 CCI 场景。

三、采集流程

华为云 CCE 集群容器内日志通过观测云标准方案 DataKit Operator 的方式采集,而弹性到 CCI 的日志通过云原生插件采集 Otel Collector 并导出到观测云 DataKit 服务,最终展示在观测云控制台,如下流程图:

四、配置步骤

步骤 1:CCE 集群弹性到 CCI Demo 搭建

  • 请自行创建 CCE 集群,并创建应用,测试可强制调度到 CCI,如下图:

sp-demo2.yaml

css 复制代码
kind: Deployment

apiVersion: apps/v1

metadata:

  name: sp-demo2

  namespace: default

  uid: 403dd3e0-8591-44d8-bd7f-0c8585acb26d

  resourceVersion: '295573'

  generation: 1

  creationTimestamp: '2025-09-12T12:15:48Z'

  labels:

    appgroup: ''

    version: v1

    virtual-kubelet.io/burst-to-cci: enforce

  annotations:

    deployment.kubernetes.io/revision: '1'

    description: ''

    kubectl.kubernetes.io/last-applied-configuration: >

      {"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{"deployment.kubernetes.io/revision":"5","description":"","workload.cce.io/swr-version":"[{"version":"Private

      Edition"}]"},"labels":{"appgroup":"","version":"v1","virtual-kubelet.io/burst-to-cci":"enforce"},"name":"sp-demo2","namespace":"default"},"spec":{"progressDeadlineSeconds":600,"replicas":1,"revisionHistoryLimit":10,"selector":{"matchLabels":{"app":"sp-demo2","version":"v1"}},"strategy":{"rollingUpdate":{"maxSurge":"25%","maxUnavailable":"25%"},"type":"RollingUpdate"},"template":{"metadata":{"labels":{"app":"sp-demo2","version":"v1"}},"spec":{"containers":[{"env":[{"name":"PAAS_APP_NAME","value":"sp-demo2"},{"name":"PAAS_NAMESPACE","value":"default"},{"name":"PAAS_PROJECT_ID","value":"bacc65fb662f435dab3acda49acae0c9"}],"image":"swr.cn-north-4.myhuaweicloud.com/liurui_bj/springboot-server:openj8","imagePullPolicy":"IfNotPresent","name":"container-1","resources":{"limits":{"cpu":"250m","memory":"512Mi"},"requests":{"cpu":"250m","memory":"512Mi"}},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File"}],"dnsPolicy":"ClusterFirst","imagePullSecrets":[{"name":"default-secret"}],"restartPolicy":"Always","schedulerName":"default-scheduler","securityContext":{},"terminationGracePeriodSeconds":30,"tolerations":[{"effect":"NoExecute","key":"node.kubernetes.io/not-ready","operator":"Exists","tolerationSeconds":300},{"effect":"NoExecute","key":"node.kubernetes.io/unreachable","operator":"Exists","tolerationSeconds":300}]}}}}

    workload.cce.io/swr-version: '[{"version":"Private Edition"}]'

  managedFields:

    - manager: kubectl-client-side-apply

      operation: Update

      apiVersion: apps/v1

      time: '2025-09-12T12:15:48Z'

      fieldsType: FieldsV1

      fieldsV1:

        f:metadata:

          f:annotations:

            .: {}

            f:description: {}

            f:kubectl.kubernetes.io/last-applied-configuration: {}

            f:workload.cce.io/swr-version: {}

          f:labels:

            .: {}

            f:appgroup: {}

            f:version: {}

            f:virtual-kubelet.io/burst-to-cci: {}

        f:spec:

          f:progressDeadlineSeconds: {}

          f:replicas: {}

          f:revisionHistoryLimit: {}

          f:selector: {}

          f:strategy:

            f:rollingUpdate:

              .: {}

              f:maxSurge: {}

              f:maxUnavailable: {}

            f:type: {}

          f:template:

            f:metadata:

              f:labels:

                .: {}

                f:app: {}

                f:version: {}

            f:spec:

              f:containers:

                k:{"name":"container-1"}:

                  .: {}

                  f:env:

                    .: {}

                    k:{"name":"PAAS_APP_NAME"}:

                      .: {}

                      f:name: {}

                      f:value: {}

                    k:{"name":"PAAS_NAMESPACE"}:

                      .: {}

                      f:name: {}

                      f:value: {}

                    k:{"name":"PAAS_PROJECT_ID"}:

                      .: {}

                      f:name: {}

                      f:value: {}

                  f:image: {}

                  f:imagePullPolicy: {}

                  f:name: {}

                  f:resources:

                    .: {}

                    f:limits:

                      .: {}

                      f:cpu: {}

                      f:memory: {}

                    f:requests:

                      .: {}

                      f:cpu: {}

                      f:memory: {}

                  f:terminationMessagePath: {}

                  f:terminationMessagePolicy: {}

              f:dnsPolicy: {}

              f:imagePullSecrets:

                .: {}

                k:{"name":"default-secret"}: {}

              f:restartPolicy: {}

              f:schedulerName: {}

              f:securityContext: {}

              f:terminationGracePeriodSeconds: {}

              f:tolerations: {}

    - manager: kube-controller-manager

      operation: Update

      apiVersion: apps/v1

      time: '2025-09-12T12:16:19Z'

      fieldsType: FieldsV1

      fieldsV1:

        f:metadata:

          f:annotations:

            f:deployment.kubernetes.io/revision: {}

        f:status:

          f:availableReplicas: {}

          f:conditions:

            .: {}

            k:{"type":"Available"}:

              .: {}

              f:lastTransitionTime: {}

              f:lastUpdateTime: {}

              f:message: {}

              f:reason: {}

              f:status: {}

              f:type: {}

            k:{"type":"Progressing"}:

              .: {}

              f:lastTransitionTime: {}

              f:lastUpdateTime: {}

              f:message: {}

              f:reason: {}

              f:status: {}

              f:type: {}

          f:observedGeneration: {}

          f:readyReplicas: {}

          f:replicas: {}

          f:updatedReplicas: {}

      subresource: status

spec:

  replicas: 1

  selector:

    matchLabels:

      app: sp-demo2

      version: v1

  template:

    metadata:

      creationTimestamp: null

      labels:

        app: sp-demo2

        version: v1

    spec:

      containers:

        - name: container-1

          image: swr.cn-north-4.myhuaweicloud.com/liurui_bj/springboot-server:openj8

          env:

            - name: PAAS_APP_NAME

              value: sp-demo2

            - name: PAAS_NAMESPACE

              value: default

            - name: PAAS_PROJECT_ID

              value: bacc65fb662f435dab3acda49acae0c9

          resources:

            limits:

              cpu: 250m

              memory: 512Mi

            requests:

              cpu: 250m

              memory: 512Mi

          terminationMessagePath: /dev/termination-log

          terminationMessagePolicy: File

          imagePullPolicy: IfNotPresent

      restartPolicy: Always

      terminationGracePeriodSeconds: 30

      dnsPolicy: ClusterFirst

      securityContext: {}

      imagePullSecrets:

        - name: default-secret

      schedulerName: default-scheduler

      tolerations:

        - key: node.kubernetes.io/not-ready

          operator: Exists

          effect: NoExecute

          tolerationSeconds: 300

        - key: node.kubernetes.io/unreachable

          operator: Exists

          effect: NoExecute

          tolerationSeconds: 300

  strategy:

    type: RollingUpdate

    rollingUpdate:

      maxUnavailable: 25%

      maxSurge: 25%

  revisionHistoryLimit: 10

  progressDeadlineSeconds: 600

status:

  observedGeneration: 1

  replicas: 1

  updatedReplicas: 1

  readyReplicas: 1

  availableReplicas: 1

  conditions:

    - type: Available

      status: 'True'

      lastUpdateTime: '2025-09-12T12:16:19Z'

      lastTransitionTime: '2025-09-12T12:16:19Z'

      reason: MinimumReplicasAvailable

      message: Deployment has minimum availability.

    - type: Progressing

      status: 'True'

      lastUpdateTime: '2025-09-12T12:16:19Z'

      lastTransitionTime: '2025-09-12T12:15:48Z'

      reason: NewReplicaSetAvailable

      message: ReplicaSet "sp-demo2-7d9cd96c44" has successfully progressed.
  • 查看 CCI 节点运行的 pod :
  • 本次要采集的 CCI 容器内日志为 server.log,目录如下:

步骤 2:在 CCE 安装云原生日志采集插件

  • 在 CCE 插件中心安装云原生日志采集插件,实例规格自定义配置
  • 在日志中心创建 CCI 日志采集策略
  • 华为云 LTS 日志采集展示

步骤 3:在 CCE 集群部署 DataKit

  • 通过 kubectl apply -f datakit.yaml 命令实现在华为云 CCE 的的一个 Daemonset 部署,采集器要开启 opentelemetry 采集器,并通过亲和性设置不让 DataKit 调度到虚拟节点

datakit.yaml

css 复制代码
kind: DaemonSet
apiVersion: apps/v1
metadata:
  name: datakit
  namespace: datakit
  uid: 122c1472-03cd-4ec6-a684-0384e40b011c
  resourceVersion: '5351437'
  generation: 2
  creationTimestamp: '2025-09-16T10:45:45Z'
  labels:
    app: daemonset-datakit
  annotations:
    deprecated.daemonset.template.generation: '2'
    kubectl.kubernetes.io/last-applied-configuration: >
      {"apiVersion":"apps/v1","kind":"DaemonSet","metadata":{"annotations":{},"labels":{"app":"daemonset-datakit"},"name":"datakit","namespace":"datakit"},"spec":{"revisionHistoryLimit":10,"selector":{"matchLabels":{"app":"daemonset-datakit"}},"template":{"metadata":{"labels":{"app":"daemonset-datakit"}},"spec":{"containers":[{"env":[{"name":"POD_NAME","valueFrom":{"fieldRef":{"fieldPath":"metadata.name"}}},{"name":"ENV_K8S_NODE_IP","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"status.hostIP"}}},{"name":"ENV_K8S_NODE_NAME","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"spec.nodeName"}}},{"name":"ENV_DATAWAY","value":"https://openway.guance.com?token=tkn_3a0052c9f6d3498c8ce9ca0988fd9c82"},{"name":"ENV_CLUSTER_NAME_K8S","value":"cce"},{"name":"ENV_GLOBAL_HOST_TAGS","value":"host=__datakit_hostname,host_ip=__datakit_ip"},{"name":"ENV_GLOBAL_ELECTION_TAGS","value":""},{"name":"ENV_DEFAULT_ENABLED_INPUTS","value":"statsd,dk,cpu,disk,diskio,mem,swap,system,hostobject,net,host_processes,container,kubernetesprometheus,logfwdserver,opentelemetry"},{"name":"ENV_ENABLE_ELECTION","value":"enable"},{"name":"ENV_INPUT_CONTAINER_ENABLE_POD_METRIC","value":"true"},{"name":"ENV_HTTP_LISTEN","value":"0.0.0.0:9529"},{"name":"ENV_INPUT_OTEL_GRPC","value":"{"addr":
      "0.0.0.0:4317"}"},{"name":"HOST_PROC","value":"/rootfs/proc"},{"name":"HOST_SYS","value":"/rootfs/sys"},{"name":"HOST_ETC","value":"/rootfs/etc"},{"name":"HOST_VAR","value":"/rootfs/var"},{"name":"HOST_RUN","value":"/rootfs/run"},{"name":"HOST_DEV","value":"/rootfs/dev"},{"name":"HOST_ROOT","value":"/rootfs"}],"image":"swr.cn-north-4.myhuaweicloud.com/liurui_bj/datakit:1.79.0","imagePullPolicy":"IfNotPresent","name":"datakit","ports":[{"containerPort":9529,"hostPort":9529,"name":"http-port","protocol":"TCP"},{"containerPort":8125,"hostPort":8125,"name":"statsd-port","protocol":"UDP"},{"containerPort":4317,"hostPort":4317,"name":"otel-grpc-port","protocol":"TCP"},{"containerPort":9533,"hostPort":9533,"name":"logfwd-port","protocol":"TCP"}],"resources":{"limits":{"cpu":"500m","memory":"1Gi"},"requests":{"cpu":"200m","memory":"128Mi"}},"securityContext":{"privileged":true},"volumeMounts":[{"mountPath":"/usr/local/datakit/cache","name":"cache","readOnly":false},{"mountPath":"/rootfs","mountPropagation":"HostToContainer","name":"rootfs"},{"mountPath":"/var/run","mountPropagation":"HostToContainer","name":"run"},{"mountPath":"/sys/kernel/debug","name":"debugfs"},{"mountPath":"/var/lib/containerd/container_logs","name":"container-logs"},{"mountPath":"/usr/local/datakit/conf.d/kubernetesprometheus/kubelet.conf","name":"datakit-conf","subPath":"kubelet.conf"}],"workingDir":"/usr/local/datakit"}],"dnsPolicy":"ClusterFirstWithHostNet","hostIPC":true,"hostNetwork":true,"hostPID":true,"restartPolicy":"Always","serviceAccount":"datakit","serviceAccountName":"datakit","tolerations":[{"operator":"Exists"}],"volumes":[{"configMap":{"name":"datakit-conf"},"name":"datakit-conf"},{"hostPath":{"path":"/"},"name":"rootfs"},{"hostPath":{"path":"/var/run"},"name":"run"},{"hostPath":{"path":"/sys/kernel/debug"},"name":"debugfs"},{"hostPath":{"path":"/root/datakit_cache"},"name":"cache"},{"hostPath":{"path":"/var/lib/containerd/container_logs"},"name":"container-logs"}]}},"updateStrategy":{"rollingUpdate":{"maxUnavailable":1},"type":"RollingUpdate"}}}
  managedFields:
    - manager: kubectl-client-side-apply
      operation: Update
      apiVersion: apps/v1
      time: '2025-09-16T10:45:45Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:annotations:
            .: {}
            f:deprecated.daemonset.template.generation: {}
            f:kubectl.kubernetes.io/last-applied-configuration: {}
          f:labels:
            .: {}
            f:app: {}
        f:spec:
          f:revisionHistoryLimit: {}
          f:selector: {}
          f:template:
            f:metadata:
              f:labels:
                .: {}
                f:app: {}
            f:spec:
              f:containers:
                k:{"name":"datakit"}:
                  .: {}
                  f:env:
                    .: {}
                    k:{"name":"ENV_CLUSTER_NAME_K8S"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"ENV_DATAWAY"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"ENV_DEFAULT_ENABLED_INPUTS"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"ENV_ENABLE_ELECTION"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"ENV_GLOBAL_ELECTION_TAGS"}:
                      .: {}
                      f:name: {}
                    k:{"name":"ENV_GLOBAL_HOST_TAGS"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"ENV_HTTP_LISTEN"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"ENV_INPUT_CONTAINER_ENABLE_POD_METRIC"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"ENV_INPUT_OTEL_GRPC"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"ENV_K8S_NODE_IP"}:
                      .: {}
                      f:name: {}
                      f:valueFrom:
                        .: {}
                        f:fieldRef: {}
                    k:{"name":"ENV_K8S_NODE_NAME"}:
                      .: {}
                      f:name: {}
                      f:valueFrom:
                        .: {}
                        f:fieldRef: {}
                    k:{"name":"HOST_DEV"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"HOST_ETC"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"HOST_PROC"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"HOST_ROOT"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"HOST_RUN"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"HOST_SYS"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"HOST_VAR"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"POD_NAME"}:
                      .: {}
                      f:name: {}
                      f:valueFrom:
                        .: {}
                        f:fieldRef: {}
                  f:image: {}
                  f:imagePullPolicy: {}
                  f:name: {}
                  f:ports:
                    .: {}
                    k:{"containerPort":4317,"protocol":"TCP"}:
                      .: {}
                      f:containerPort: {}
                      f:hostPort: {}
                      f:name: {}
                      f:protocol: {}
                    k:{"containerPort":8125,"protocol":"UDP"}:
                      .: {}
                      f:containerPort: {}
                      f:hostPort: {}
                      f:name: {}
                      f:protocol: {}
                    k:{"containerPort":9529,"protocol":"TCP"}:
                      .: {}
                      f:containerPort: {}
                      f:hostPort: {}
                      f:name: {}
                      f:protocol: {}
                    k:{"containerPort":9533,"protocol":"TCP"}:
                      .: {}
                      f:containerPort: {}
                      f:hostPort: {}
                      f:name: {}
                      f:protocol: {}
                  f:resources:
                    .: {}
                    f:limits:
                      .: {}
                      f:cpu: {}
                      f:memory: {}
                    f:requests:
                      .: {}
                      f:cpu: {}
                      f:memory: {}
                  f:securityContext:
                    .: {}
                    f:privileged: {}
                  f:terminationMessagePath: {}
                  f:terminationMessagePolicy: {}
                  f:volumeMounts:
                    .: {}
                    k:{"mountPath":"/rootfs"}:
                      .: {}
                      f:mountPath: {}
                      f:mountPropagation: {}
                      f:name: {}
                    k:{"mountPath":"/sys/kernel/debug"}:
                      .: {}
                      f:mountPath: {}
                      f:name: {}
                    k:{"mountPath":"/usr/local/datakit/cache"}:
                      .: {}
                      f:mountPath: {}
                      f:name: {}
                    k:{"mountPath":"/usr/local/datakit/conf.d/kubernetesprometheus/kubelet.conf"}:
                      .: {}
                      f:mountPath: {}
                      f:name: {}
                      f:subPath: {}
                    k:{"mountPath":"/var/lib/containerd/container_logs"}:
                      .: {}
                      f:mountPath: {}
                      f:name: {}
                    k:{"mountPath":"/var/run"}:
                      .: {}
                      f:mountPath: {}
                      f:mountPropagation: {}
                      f:name: {}
                  f:workingDir: {}
              f:dnsPolicy: {}
              f:hostIPC: {}
              f:hostNetwork: {}
              f:hostPID: {}
              f:restartPolicy: {}
              f:schedulerName: {}
              f:securityContext: {}
              f:serviceAccount: {}
              f:serviceAccountName: {}
              f:terminationGracePeriodSeconds: {}
              f:tolerations: {}
              f:volumes:
                .: {}
                k:{"name":"cache"}:
                  .: {}
                  f:hostPath:
                    .: {}
                    f:path: {}
                    f:type: {}
                  f:name: {}
                k:{"name":"container-logs"}:
                  .: {}
                  f:hostPath:
                    .: {}
                    f:path: {}
                    f:type: {}
                  f:name: {}
                k:{"name":"datakit-conf"}:
                  .: {}
                  f:configMap:
                    .: {}
                    f:defaultMode: {}
                    f:name: {}
                  f:name: {}
                k:{"name":"debugfs"}:
                  .: {}
                  f:hostPath:
                    .: {}
                    f:path: {}
                    f:type: {}
                  f:name: {}
                k:{"name":"rootfs"}:
                  .: {}
                  f:hostPath:
                    .: {}
                    f:path: {}
                    f:type: {}
                  f:name: {}
                k:{"name":"run"}:
                  .: {}
                  f:hostPath:
                    .: {}
                    f:path: {}
                    f:type: {}
                  f:name: {}
          f:updateStrategy:
            f:rollingUpdate:
              .: {}
              f:maxSurge: {}
              f:maxUnavailable: {}
            f:type: {}
    - manager: cfe-apiserver
      operation: Update
      apiVersion: apps/v1
      time: '2025-09-19T06:28:11Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:spec:
          f:template:
            f:spec:
              f:affinity:
                .: {}
                f:nodeAffinity:
                  .: {}
                  f:requiredDuringSchedulingIgnoredDuringExecution: {}
    - manager: kube-controller-manager
      operation: Update
      apiVersion: apps/v1
      time: '2025-09-19T06:28:19Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:status:
          f:currentNumberScheduled: {}
          f:desiredNumberScheduled: {}
          f:numberAvailable: {}
          f:numberMisscheduled: {}
          f:numberReady: {}
          f:observedGeneration: {}
          f:updatedNumberScheduled: {}
      subresource: status
spec:
  selector:
    matchLabels:
      app: daemonset-datakit
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: daemonset-datakit
    spec:
      volumes:
        - name: datakit-conf
          configMap:
            name: datakit-conf
            defaultMode: 420
        - name: rootfs
          hostPath:
            path: /
            type: ''
        - name: run
          hostPath:
            path: /var/run
            type: ''
        - name: debugfs
          hostPath:
            path: /sys/kernel/debug
            type: ''
        - name: cache
          hostPath:
            path: /root/datakit_cache
            type: ''
        - name: container-logs
          hostPath:
            path: /var/lib/containerd/container_logs
            type: ''
      containers:
        - name: datakit
          image: swr.cn-north-4.myhuaweicloud.com/liurui_bj/datakit:1.79.0
          workingDir: /usr/local/datakit
          ports:
            - name: http-port
              hostPort: 9529
              containerPort: 9529
              protocol: TCP
            - name: statsd-port
              hostPort: 8125
              containerPort: 8125
              protocol: UDP
            - name: otel-grpc-port
              hostPort: 4317
              containerPort: 4317
              protocol: TCP
            - name: logfwd-port
              hostPort: 9533
              containerPort: 9533
              protocol: TCP
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.name
            - name: ENV_K8S_NODE_IP
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: status.hostIP
            - name: ENV_K8S_NODE_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: spec.nodeName
            - name: ENV_DATAWAY
              value: https://openway.guance.com?token=tkn_3a0052c9f6d3498c8ce9ca0988fd9c82
            - name: ENV_CLUSTER_NAME_K8S
              value: cce
            - name: ENV_GLOBAL_HOST_TAGS
              value: host=__datakit_hostname,host_ip=__datakit_ip
            - name: ENV_GLOBAL_ELECTION_TAGS
            - name: ENV_DEFAULT_ENABLED_INPUTS
              value: statsd,dk,cpu,disk,diskio,mem,swap,system,hostobject,net,host_processes,container,kubernetesprometheus,logfwdserver,opentelemetry
            - name: ENV_ENABLE_ELECTION
              value: enable
            - name: ENV_INPUT_CONTAINER_ENABLE_POD_METRIC
              value: 'true'
            - name: ENV_HTTP_LISTEN
              value: 0.0.0.0:9529
            - name: ENV_INPUT_OTEL_GRPC
              value: '{"addr": "0.0.0.0:4317"}'
            - name: HOST_PROC
              value: /rootfs/proc
            - name: HOST_SYS
              value: /rootfs/sys
            - name: HOST_ETC
              value: /rootfs/etc
            - name: HOST_VAR
              value: /rootfs/var
            - name: HOST_RUN
              value: /rootfs/run
            - name: HOST_DEV
              value: /rootfs/dev
            - name: HOST_ROOT
              value: /rootfs
          resources:
            limits:
              cpu: 500m
              memory: 1Gi
            requests:
              cpu: 200m
              memory: 128Mi
          volumeMounts:
            - name: cache
              mountPath: /usr/local/datakit/cache
            - name: rootfs
              mountPath: /rootfs
              mountPropagation: HostToContainer
            - name: run
              mountPath: /var/run
              mountPropagation: HostToContainer
            - name: debugfs
              mountPath: /sys/kernel/debug
            - name: container-logs
              mountPath: /var/lib/containerd/container_logs
            - name: datakit-conf
              mountPath: /usr/local/datakit/conf.d/kubernetesprometheus/kubelet.conf
              subPath: kubelet.conf
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: IfNotPresent
          securityContext:
            privileged: true
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      dnsPolicy: ClusterFirstWithHostNet
      serviceAccountName: datakit
      serviceAccount: datakit
      hostNetwork: true
      hostPID: true
      hostIPC: true
      securityContext: {}
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: bursting.cci.io/node-type
                    operator: NotIn
                    values:
                      - virtual-kubelet
      schedulerName: default-scheduler
      tolerations:
        - operator: Exists
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
      maxSurge: 0
  revisionHistoryLimit: 10
status:
  currentNumberScheduled: 2
  numberMisscheduled: 0
  desiredNumberScheduled: 2
  numberReady: 2
  observedGeneration: 2
  updatedNumberScheduled: 2
  numberAvailable: 2
  • 进入 datakit 容器,并执行 datakit monitor 查看 opentelemetry 采集器是否开启

步骤 4:重写 Otel Collector 的采集配置

log-agent-otel-collector.yaml

css 复制代码
kind: Deployment
apiVersion: apps/v1
metadata:
  name: log-agent-otel-collector
  namespace: monitoring
  uid: c055d466-4287-4860-9ff7-d28cc036ae89
  resourceVersion: '7557223'
  generation: 3
  creationTimestamp: '2025-09-22T07:28:09Z'
  labels:
    app: log-agent-otel-collector
    app.kubernetes.io/managed-by: Helm
    release: cceaddon-log-agent
  annotations:
    deployment.kubernetes.io/revision: '3'
    kubectl.kubernetes.io/last-applied-configuration: >
      {"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{"deployment.kubernetes.io/revision":"3","meta.helm.sh/release-name":"cceaddon-log-agent","meta.helm.sh/release-namespace":"monitoring"},"creationTimestamp":"2025-09-20T19:02:18Z","generation":3,"labels":{"app":"log-agent-otel-collector","app.kubernetes.io/managed-by":"Helm","release":"cceaddon-log-agent"},"name":"log-agent-otel-collector","namespace":"monitoring","resourceVersion":"7514159","uid":"180806a1-7260-4139-989c-73945d7b1a4c"},"spec":{"minReadySeconds":5,"progressDeadlineSeconds":120,"replicas":2,"revisionHistoryLimit":10,"selector":{"matchLabels":{"app":"log-agent-otel-collector"}},"strategy":{"rollingUpdate":{"maxSurge":1,"maxUnavailable":1},"type":"RollingUpdate"},"template":{"metadata":{"annotations":{"prometheus.io/path":"/metrics","prometheus.io/port":"8888","prometheus.io/scheme":"http","prometheus.io/scrape":"true","redeploy-timestamp":"1758396245987","scheduler.alpha.kubernetes.io/tolerations":"[{"key":
      "taint.alpha.kubernetes.io/nodedown","value": "","effect": "NoExecute","operator":
      "Exists"}]"},"creationTimestamp":null,"labels":{"app":"log-agent-otel-collector","release":"cceaddon-log-agent"}},"spec":{"affinity":{"podAntiAffinity":{"preferredDuringSchedulingIgnoredDuringExecution":[{"podAffinityTerm":{"labelSelector":{"matchExpressions":[{"key":"app","operator":"In","values":["log-agent-otel-collector"]}]},"topologyKey":"topology.kubernetes.io/zone"},"weight":100}]}},"containers":[{"args":["--config=/var/paas/ot-collector/ot-collector-service.yaml"],"command":["/var/paas/otel-collector/otelcol"],"env":[{"name":"POD_IP","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"status.podIP"}}},{"name":"Region","value":"cn-north-4"},{"name":"ProjectID","value":"9e92837f567145009ad4d230c4ac2c01"},{"name":"ClusterID","value":"74e8b92f-8f80-11f0-afe1-0255ac10026c"},{"name":"ClusterName","value":"cce-cci"},{"name":"WATCH_SECRET","value":"true"},{"name":"INSECURE_SKIP_VERIFY","value":"true"},{"name":"SCENE","value":"HWS"},{"name":"AKSK_SECRET_NAME","value":"paas.elb"},{"name":"WATCH_CLUSTER_CONFIG","value":"true"},{"name":"AOM_ENDPOINT","value":"https://aom.cn-north-4.myhuaweicloud.com"},{"name":"LTS_ACCESS_ENDPOINT","value":"https://lts-access.cn-north-4.myhuaweicloud.com:8102"},{"name":"CRYPTO_ENABLE","value":"true"},{"name":"PAAS_CRYPTO_PATH","value":"/etc/cipher"}],"image":"swr.cn-north-4.myhuaweicloud.com/hwofficial/otelcol:1.7.4","imagePullPolicy":"IfNotPresent","livenessProbe":{"exec":{"command":["/bin/bash","-c","exit
      0"]},"failureThreshold":3,"initialDelaySeconds":20,"periodSeconds":20,"successThreshold":1,"timeoutSeconds":10},"name":"otel-collector","ports":[{"containerPort":8006,"protocol":"TCP"},{"containerPort":4317,"protocol":"TCP"},{"containerPort":8888,"name":"metric-port","protocol":"TCP"}],"resources":{"limits":{"cpu":"1","memory":"2Gi"},"requests":{"cpu":"200m","memory":"1Gi"}},"securityContext":{"allowPrivilegeEscalation":false,"readOnlyRootFilesystem":true,"runAsGroup":10000,"runAsUser":10000},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","volumeMounts":[{"mountPath":"/var/paas/otel-collector/conf","name":"otel-collector-config-vol","readOnly":true},{"mountPath":"/var/paas/ot-collector/ot-collector-service.yaml","name":"ot-collector-service","readOnly":true,"subPath":"ot-collector-service.yaml"},{"mountPath":"/var/paas/sys/log","name":"logpath"},{"mountPath":"/etc/cipher/root.key","name":"rootkey","readOnly":true},{"mountPath":"/etc/cipher/common_shared.key","name":"commonsharedkey","readOnly":true},{"mountPath":"/var/paas/cert","name":"cert","readOnly":true}]}],"dnsConfig":{"options":[{"name":"ndots","value":"3"}]},"dnsPolicy":"ClusterFirst","initContainers":[{"command":["/bin/sh","-c","mkdir
      -p /var/paas/sys/log/otel \u0026\u0026 chmod 750 /var/paas/sys/log/otel \u0026\u0026 chown -R 10000:10000
      /var/paas/sys/log/otel"],"image":"swr.cn-north-4.myhuaweicloud.com/hwofficial/otelcol:1.7.4","imagePullPolicy":"IfNotPresent","name":"init","resources":{"limits":{"cpu":"200m","memory":"200Mi"},"requests":{"cpu":"100m","memory":"100Mi"}},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","volumeMounts":[{"mountPath":"/var/paas/sys/log","name":"logpath"}]}],"priorityClassName":"system-cluster-critical","restartPolicy":"Always","schedulerName":"default-scheduler","securityContext":{"fsGroup":10000},"serviceAccount":"log-agent-serviceaccount","serviceAccountName":"log-agent-serviceaccount","terminationGracePeriodSeconds":30,"tolerations":[{"effect":"NoExecute","key":"node.kubernetes.io/not-ready","operator":"Exists","tolerationSeconds":30},{"effect":"NoExecute","key":"node.kubernetes.io/unreachable","operator":"Exists","tolerationSeconds":30},{"key":"role","operator":"Exists"},{"effect":"NoSchedule","key":"distribution.io/category","operator":"Equal","value":"IES"}],"volumes":[{"name":"otel-collector-config-vol","secret":{"defaultMode":384,"secretName":"log-agent-otel-collector-config"}},{"configMap":{"defaultMode":420,"items":[{"key":"ot-collector-service.yaml","path":"ot-collector-service.yaml"}],"name":"ot-collector-service"},"name":"ot-collector-service"},{"name":"cert","secret":{"defaultMode":416,"items":[{"key":"caCert","path":"caCert"},{"key":"serverCert","path":"serverCert"},{"key":"serverKey","path":"serverKey"}],"secretName":"log-agent-cert-secret"}},{"hostPath":{"path":"/var/paas/sys/log","type":""},"name":"logpath"},{"hostPath":{"path":"/var/paas/srv/kubernetes/root.key","type":""},"name":"rootkey"},{"hostPath":{"path":"/var/paas/srv/kubernetes/common_shared.key","type":""},"name":"commonsharedkey"}]}}},"status":{"conditions":[{"lastTransitionTime":"2025-09-20T19:02:18Z","lastUpdateTime":"2025-09-20T19:37:31Z","message":"ReplicaSet
      "log-agent-otel-collector-8fbf8c694" has successfully progressed.","reason":"NewReplicaSetAvailable","status":"True","type":"Progressing"},{"lastTransitionTime":"2025-09-22T06:15:38Z","lastUpdateTime":"2025-09-22T06:15:38Z","message":"Deployment does not have minimum availability.","reason":"MinimumReplicasUnavailable","status":"False","type":"Available"}],"observedGeneration":3,"replicas":2,"unavailableReplicas":2,"updatedReplicas":2}}
    meta.helm.sh/release-name: cceaddon-log-agent
    meta.helm.sh/release-namespace: monitoring
  managedFields:
    - manager: kubectl-client-side-apply
      operation: Update
      apiVersion: apps/v1
      time: '2025-09-22T07:28:09Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:annotations:
            .: {}
            f:kubectl.kubernetes.io/last-applied-configuration: {}
            f:meta.helm.sh/release-name: {}
            f:meta.helm.sh/release-namespace: {}
          f:labels:
            .: {}
            f:app: {}
            f:app.kubernetes.io/managed-by: {}
            f:release: {}
        f:spec:
          f:minReadySeconds: {}
          f:progressDeadlineSeconds: {}
          f:replicas: {}
          f:revisionHistoryLimit: {}
          f:selector: {}
          f:strategy:
            f:rollingUpdate:
              .: {}
              f:maxSurge: {}
              f:maxUnavailable: {}
            f:type: {}
          f:template:
            f:metadata:
              f:annotations:
                .: {}
                f:prometheus.io/path: {}
                f:prometheus.io/port: {}
                f:prometheus.io/scheme: {}
                f:prometheus.io/scrape: {}
                f:scheduler.alpha.kubernetes.io/tolerations: {}
              f:labels:
                .: {}
                f:app: {}
                f:release: {}
            f:spec:
              f:affinity:
                .: {}
                f:podAntiAffinity:
                  .: {}
                  f:preferredDuringSchedulingIgnoredDuringExecution: {}
              f:containers:
                k:{"name":"otel-collector"}:
                  .: {}
                  f:args: {}
                  f:command: {}
                  f:env:
                    .: {}
                    k:{"name":"AKSK_SECRET_NAME"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"AOM_ENDPOINT"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"CRYPTO_ENABLE"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"ClusterID"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"ClusterName"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"INSECURE_SKIP_VERIFY"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"LTS_ACCESS_ENDPOINT"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"PAAS_CRYPTO_PATH"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"POD_IP"}:
                      .: {}
                      f:name: {}
                      f:valueFrom:
                        .: {}
                        f:fieldRef: {}
                    k:{"name":"ProjectID"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"Region"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"SCENE"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"WATCH_CLUSTER_CONFIG"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"WATCH_SECRET"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                  f:image: {}
                  f:imagePullPolicy: {}
                  f:livenessProbe:
                    .: {}
                    f:exec:
                      .: {}
                      f:command: {}
                    f:failureThreshold: {}
                    f:initialDelaySeconds: {}
                    f:periodSeconds: {}
                    f:successThreshold: {}
                    f:timeoutSeconds: {}
                  f:name: {}
                  f:ports:
                    .: {}
                    k:{"containerPort":4317,"protocol":"TCP"}:
                      .: {}
                      f:containerPort: {}
                      f:protocol: {}
                    k:{"containerPort":8006,"protocol":"TCP"}:
                      .: {}
                      f:containerPort: {}
                      f:protocol: {}
                    k:{"containerPort":8888,"protocol":"TCP"}:
                      .: {}
                      f:containerPort: {}
                      f:name: {}
                      f:protocol: {}
                  f:resources:
                    .: {}
                    f:limits:
                      .: {}
                      f:cpu: {}
                      f:memory: {}
                    f:requests:
                      .: {}
                      f:cpu: {}
                      f:memory: {}
                  f:securityContext:
                    .: {}
                    f:allowPrivilegeEscalation: {}
                    f:readOnlyRootFilesystem: {}
                    f:runAsGroup: {}
                    f:runAsUser: {}
                  f:terminationMessagePath: {}
                  f:terminationMessagePolicy: {}
                  f:volumeMounts:
                    .: {}
                    k:{"mountPath":"/etc/cipher/common_shared.key"}:
                      .: {}
                      f:mountPath: {}
                      f:name: {}
                      f:readOnly: {}
                    k:{"mountPath":"/etc/cipher/root.key"}:
                      .: {}
                      f:mountPath: {}
                      f:name: {}
                      f:readOnly: {}
                    k:{"mountPath":"/var/paas/cert"}:
                      .: {}
                      f:mountPath: {}
                      f:name: {}
                      f:readOnly: {}
                    k:{"mountPath":"/var/paas/ot-collector/ot-collector-service.yaml"}:
                      .: {}
                      f:mountPath: {}
                      f:name: {}
                      f:readOnly: {}
                      f:subPath: {}
                    k:{"mountPath":"/var/paas/otel-collector/conf"}:
                      .: {}
                      f:mountPath: {}
                      f:name: {}
                      f:readOnly: {}
                    k:{"mountPath":"/var/paas/sys/log"}:
                      .: {}
                      f:mountPath: {}
                      f:name: {}
              f:dnsConfig:
                .: {}
                f:options: {}
              f:dnsPolicy: {}
              f:initContainers:
                .: {}
                k:{"name":"init"}:
                  .: {}
                  f:command: {}
                  f:image: {}
                  f:imagePullPolicy: {}
                  f:name: {}
                  f:resources:
                    .: {}
                    f:limits:
                      .: {}
                      f:cpu: {}
                      f:memory: {}
                    f:requests:
                      .: {}
                      f:cpu: {}
                      f:memory: {}
                  f:terminationMessagePath: {}
                  f:terminationMessagePolicy: {}
                  f:volumeMounts:
                    .: {}
                    k:{"mountPath":"/var/paas/sys/log"}:
                      .: {}
                      f:mountPath: {}
                      f:name: {}
              f:priorityClassName: {}
              f:restartPolicy: {}
              f:schedulerName: {}
              f:securityContext:
                .: {}
                f:fsGroup: {}
              f:serviceAccount: {}
              f:serviceAccountName: {}
              f:terminationGracePeriodSeconds: {}
              f:tolerations: {}
              f:volumes:
                .: {}
                k:{"name":"cert"}:
                  .: {}
                  f:name: {}
                  f:secret:
                    .: {}
                    f:defaultMode: {}
                    f:items: {}
                    f:secretName: {}
                k:{"name":"commonsharedkey"}:
                  .: {}
                  f:hostPath:
                    .: {}
                    f:path: {}
                    f:type: {}
                  f:name: {}
                k:{"name":"logpath"}:
                  .: {}
                  f:hostPath:
                    .: {}
                    f:path: {}
                    f:type: {}
                  f:name: {}
                k:{"name":"ot-collector-service"}:
                  .: {}
                  f:configMap:
                    .: {}
                    f:defaultMode: {}
                    f:items: {}
                    f:name: {}
                  f:name: {}
                k:{"name":"otel-collector-config-vol"}:
                  .: {}
                  f:name: {}
                  f:secret:
                    .: {}
                    f:defaultMode: {}
                    f:secretName: {}
                k:{"name":"rootkey"}:
                  .: {}
                  f:hostPath:
                    .: {}
                    f:path: {}
                    f:type: {}
                  f:name: {}
    - manager: cfe-apiserver
      operation: Update
      apiVersion: apps/v1
      time: '2025-09-22T07:40:23Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:spec:
          f:template:
            f:metadata:
              f:annotations:
                f:redeploy-timestamp: {}
    - manager: kube-controller-manager
      operation: Update
      apiVersion: apps/v1
      time: '2025-09-22T07:40:31Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:annotations:
            f:deployment.kubernetes.io/revision: {}
        f:status:
          f:availableReplicas: {}
          f:conditions:
            .: {}
            k:{"type":"Available"}:
              .: {}
              f:lastTransitionTime: {}
              f:lastUpdateTime: {}
              f:message: {}
              f:reason: {}
              f:status: {}
              f:type: {}
            k:{"type":"Progressing"}:
              .: {}
              f:lastTransitionTime: {}
              f:lastUpdateTime: {}
              f:message: {}
              f:reason: {}
              f:status: {}
              f:type: {}
          f:observedGeneration: {}
          f:readyReplicas: {}
          f:replicas: {}
          f:updatedReplicas: {}
      subresource: status
spec:
  replicas: 2
  selector:
    matchLabels:
      app: log-agent-otel-collector
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: log-agent-otel-collector
        release: cceaddon-log-agent
      annotations:
        prometheus.io/path: /metrics
        prometheus.io/port: '8888'
        prometheus.io/scheme: http
        prometheus.io/scrape: 'true'
        redeploy-timestamp: '1758526823089'
        scheduler.alpha.kubernetes.io/tolerations: '[{"key": "taint.alpha.kubernetes.io/nodedown","value": "","effect": "NoExecute","operator": "Exists"}]'
    spec:
      volumes:
        - name: otel-collector-config-vol
          secret:
            secretName: log-agent-otel-collector-config
            defaultMode: 384
        - name: ot-collector-service
          configMap:
            name: ot-collector-service
            items:
              - key: ot-collector-service.yaml
                path: ot-collector-service.yaml
            defaultMode: 420
        - name: cert
          secret:
            secretName: log-agent-cert-secret
            items:
              - key: caCert
                path: caCert
              - key: serverCert
                path: serverCert
              - key: serverKey
                path: serverKey
            defaultMode: 416
        - name: logpath
          hostPath:
            path: /var/paas/sys/log
            type: ''
        - name: rootkey
          hostPath:
            path: /var/paas/srv/kubernetes/root.key
            type: ''
        - name: commonsharedkey
          hostPath:
            path: /var/paas/srv/kubernetes/common_shared.key
            type: ''
      initContainers:
        - name: init
          image: swr.cn-north-4.myhuaweicloud.com/hwofficial/otelcol:1.7.4
          command:
            - /bin/sh
            - '-c'
            - mkdir -p /var/paas/sys/log/otel && chmod 750 /var/paas/sys/log/otel && chown -R 10000:10000 /var/paas/sys/log/otel
          resources:
            limits:
              cpu: 200m
              memory: 200Mi
            requests:
              cpu: 100m
              memory: 100Mi
          volumeMounts:
            - name: logpath
              mountPath: /var/paas/sys/log
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: IfNotPresent
      containers:
        - name: otel-collector
          image: swr.cn-north-4.myhuaweicloud.com/hwofficial/otelcol:1.7.4
          command:
            - /var/paas/otel-collector/otelcol
          args:
            - '--config=/var/paas/ot-collector/ot-collector-service.yaml'
          ports:
            - containerPort: 8006
              protocol: TCP
            - containerPort: 4317
              protocol: TCP
            - name: metric-port
              containerPort: 8888
              protocol: TCP
          env:
            - name: POD_IP
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: status.podIP
            - name: Region
              value: cn-north-4
            - name: ProjectID
              value: 9e92837f567145009ad4d230c4ac2c01
            - name: ClusterID
              value: 74e8b92f-8f80-11f0-afe1-0255ac10026c
            - name: ClusterName
              value: cce-cci
            - name: WATCH_SECRET
              value: 'true'
            - name: INSECURE_SKIP_VERIFY
              value: 'true'
            - name: SCENE
              value: HWS
            - name: AKSK_SECRET_NAME
              value: paas.elb
            - name: WATCH_CLUSTER_CONFIG
              value: 'true'
            - name: AOM_ENDPOINT
              value: https://aom.cn-north-4.myhuaweicloud.com
            - name: LTS_ACCESS_ENDPOINT
              value: https://lts-access.cn-north-4.myhuaweicloud.com:8102
            - name: CRYPTO_ENABLE
              value: 'true'
            - name: PAAS_CRYPTO_PATH
              value: /etc/cipher
          resources:
            limits:
              cpu: '1'
              memory: 2Gi
            requests:
              cpu: 200m
              memory: 1Gi
          volumeMounts:
            - name: otel-collector-config-vol
              readOnly: true
              mountPath: /var/paas/otel-collector/conf
            - name: ot-collector-service
              readOnly: true
              mountPath: /var/paas/ot-collector/ot-collector-service.yaml
              subPath: ot-collector-service.yaml
            - name: logpath
              mountPath: /var/paas/sys/log
            - name: rootkey
              readOnly: true
              mountPath: /etc/cipher/root.key
            - name: commonsharedkey
              readOnly: true
              mountPath: /etc/cipher/common_shared.key
            - name: cert
              readOnly: true
              mountPath: /var/paas/cert
          livenessProbe:
            exec:
              command:
                - /bin/bash
                - '-c'
                - exit 0
            initialDelaySeconds: 20
            timeoutSeconds: 10
            periodSeconds: 20
            successThreshold: 1
            failureThreshold: 3
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: IfNotPresent
          securityContext:
            runAsUser: 10000
            runAsGroup: 10000
            readOnlyRootFilesystem: true
            allowPrivilegeEscalation: false
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      dnsPolicy: ClusterFirst
      serviceAccountName: log-agent-serviceaccount
      serviceAccount: log-agent-serviceaccount
      securityContext:
        fsGroup: 10000
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              podAffinityTerm:
                labelSelector:
                  matchExpressions:
                    - key: app
                      operator: In
                      values:
                        - log-agent-otel-collector
                topologyKey: topology.kubernetes.io/zone
      schedulerName: default-scheduler
      tolerations:
        - key: node.kubernetes.io/not-ready
          operator: Exists
          effect: NoExecute
          tolerationSeconds: 30
        - key: node.kubernetes.io/unreachable
          operator: Exists
          effect: NoExecute
          tolerationSeconds: 30
        - key: role
          operator: Exists
        - key: distribution.io/category
          operator: Equal
          value: IES
          effect: NoSchedule
      priorityClassName: system-cluster-critical
      dnsConfig:
        options:
          - name: ndots
            value: '3'
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
      maxSurge: 1
  minReadySeconds: 5
  revisionHistoryLimit: 10
  progressDeadlineSeconds: 120
status:
  observedGeneration: 3
  replicas: 2
  updatedReplicas: 2
  readyReplicas: 2
  availableReplicas: 2
  conditions:
    - type: Available
      status: 'True'
      lastUpdateTime: '2025-09-22T07:28:16Z'
      lastTransitionTime: '2025-09-22T07:28:16Z'
      reason: MinimumReplicasAvailable
      message: Deployment has minimum availability.
    - type: Progressing
      status: 'True'
      lastUpdateTime: '2025-09-22T07:40:31Z'
      lastTransitionTime: '2025-09-22T07:28:09Z'
      reason: NewReplicaSetAvailable
      message: ReplicaSet "log-agent-otel-collector-5cfd6f4c7c" has successfully progressed.
  • 为避免配置覆盖以及确保配置生效,指定 Otel Collector 启动加载生效的配置
  • Otel Collector 挂载新的配置
  • 关闭健康检查
  • 若要实现 LTS 和观测云的数据双写,挂载的配置如下:
yaml 复制代码
exporters:
  aom/default-event-aom:
    endpoint: https://aom.cn-north-4.myhuaweicloud.com
    events:
    - name: DeleteNodeWithNoServer
      name_cn: 废弃节点清理
  ...
  lts/default-stdout:
    compress_type: gzip
    endpoint: https://lts-access.cn-north-4.myhuaweicloud.com:8102
    log_type: log
    lts_group_id: d6b393b8-484f-4835-ba9f-xxxxx
    lts_stream_id: 8e02106f-8aeb-4da5-a5e1-xxxxx
  otlphttp:
    endpoint: http://datakit-service.datakit:9529/otel
    tls:
      insecure: true          
processors:
  batch/default-event:
    send_batch_max_size: 1000
    send_batch_size: 500
    timeout: 1000000000
  ...
  filter/cci-log:
    logs:
      exclude: {}
      include:
        match_type: strict
        record_attributes:
        - key: logconfig
          value: cci-log
  filter/datakit:
    logs:
      exclude: {}
      include:
        match_type: strict
        record_attributes:
        - key: logconfig
          value: datakit
service:
  pipelines:
    logs/cci-log:
      exporters:
      - lts/cci-log
      - otlphttp
  ...
  • 挂载的配置若是只写到观测云,配置如下:
yaml 复制代码
exporters:
  otlphttp:
    endpoint: http://datakit-service.datakit:9529/otel
    tls:
      insecure: true
processors:
  batch/logs:
    send_batch_max_size: 2000
    send_batch_size: 2000
  filter/cci-log:
    logs:
      exclude: {}
      include:
        match_type: strict
        record_attributes:
        - key: logconfig
          value: cci-log
receivers:
  fluentforward:
    endpoint: ${POD_IP}:8006
    tls:
      cert_file: /var/paas/cert/serverCert
      client_ca_file: /var/paas/cert/caCert
      key_file: /var/paas/cert/serverKey
  k8s_events: {}
service:
  pipelines:
    logs/cci-log:
      exporters:
      - otlphttp
      processors:
      - filter/cci-log
      - batch/logs
      receivers:
      - fluentforward
  telemetry:
    logs: {}
    metrics:
      address: ${POD_IP}:8888
      level: basic

步骤 5:容器 demo 发起请求,产生日志

步骤 6:在观测云验证日志接入

  • 登录观测云控制台 → 日志查看器 ,可以看到相关日志已经被采集到了观测云。
相关推荐
CodeLongBear4 小时前
Python数据分析: 数据可视化入门:Matplotlib基础操作与多坐标系实战
python·信息可视化·数据分析
程序员鱼皮9 小时前
10个免费的网站分析工具,竟然比付费的更香?
后端·程序员·数据分析
咚咚王者1 天前
人工智能之数据分析 numpy:第十二章 数据持久化
人工智能·数据分析·numpy
c***42101 天前
爬虫基础之爬取某基金网站+数据分析
爬虫·数据挖掘·数据分析
大云计算机毕设1 天前
【2026计算机毕设选题】计算机毕设全新推荐项目选题指南(70+精选热门方向)
spring·数据分析·毕业设计·课程设计·毕设
图灵信徒1 天前
R语言第七章线性回归模型
数据挖掘·数据分析·r语言·线性回归
rgb2gray2 天前
增强城市数据分析:多密度区域的自适应分区框架
大数据·python·机器学习·语言模型·数据挖掘·数据分析·llm
学术小白人2 天前
会议第一轮投稿!2026年物联网、数据科学与先进计算国际学术会议(IDSAC2026)
人工智能·物联网·数据分析·能源·制造·教育·rdlink研发家
X***E4633 天前
前端数据分析应用
前端·数据挖掘·数据分析