K8S离线部署Nacos集群【Oracle作外部数据源】

cs阿坤dn2024-12-06 12:28

一、前言

  由于公司的要求下要使Nacos集群以Oracle作为外部数据源，前期咱们已经阐述了如何在本地搭建（Nacos集群搭建【Oracle作外部数据源】），本次将带领大家在k8s上部署Nacos集群并以Oracle作为外部数据源。

二、软件包

说明：nacos-server-oracle.tar该镜像包经过本人dockerfile构建修改、加入了oracle相关变量、参数、以及支持oracle 11g的二次编译nacos-server.jar程序包。

三、环境

  • k8s1.23.1
  • nacos-server2.3.2
  • Oracle 11g RAC集群

四、部署

1.创建命名空间

bash 复制代码 
ls
cat 00-ns.yaml 

apiVersion: v1
kind: Namespace
metadata:
  name: nacoscluster
  labels:
    app: nacoscluster
    
kubectl apply -f 00-ns.yaml

2.创建存储类

bash 复制代码 
cat 00-sc.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: local-storage2
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

kubectl apply -f 00-sc.yaml

3.创建pv卷

bash 复制代码 
 cat pv.yaml 
apiVersion: v1
kind: PersistentVolume
metadata:
  name: local-storage-nacos-pv-0
  namespace: nacoscluster     # 空间名称
  labels:
    name: local-storage-nacos-pv-0
spec:
  capacity:
    storage: 100Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage2
  local:
    path: /srv/nacos_data #存储卷的实际位置，与nfs暴露的文件地址相同
  nodeAffinity:
    required:
      nodeSelectorTerms:        # 节点选择
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - node3            # 根据自己节点名称
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: local-storage-nacos-pv-1
  namespace: nacoscluster     # 空间名称
  labels:
    name: local-storage-nacos-pv-1
spec:
  capacity:
    storage: 100Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage2
  local:
    path: /srv/nacos_data #存储卷的实际位置，与nfs暴露的文件地址相同
  nodeAffinity:
    required:
      nodeSelectorTerms:        # 节点选择
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - node4            # 根据自己节点名称
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: local-storage-nacos-pv-2
  namespace: nacoscluster     # 空间名称
  labels:
    name: local-storage-nacos-pv-2
spec:
  capacity:
    storage: 100Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage2
  local:
    path: /srv/nacos_data #存储卷的实际位置，与nfs暴露的文件地址相同
  nodeAffinity:
    required:
      nodeSelectorTerms:        # 节点选择
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - node5            # 根据自己节点名称

kubectl apply -f  pv.yaml

注意：需要在对应节点(例：node3、node4、node5)创建该目录/srv/nacos_data

3.给相应节点打标签

bash 复制代码 
# 用于nacos容器绑定到指定节点（可选）
kubectl label nodes node3 app=nacos
kubectl label nodes node4 app=nacos
kubectl label nodes node5 app=nacos

4.创建nacos集群

bash 复制代码 
cat nacos-pvc.yaml

###使用自建数据库；使用Ingress发布配置后台###
---
apiVersion: v1
kind: Service
metadata:
  name: nacos-headless
  namespace: nacoscluster
  labels:
    app: nacos-headless
spec:
  type: ClusterIP
  clusterIP: None
  ports:
    - name: tcp-8848
      protocol: TCP
      port: 8848
      targetPort: 8848
    - name: tcp-9848
      protocol: TCP
      port: 9848
      targetPort: 9848
    - name: tcp-9849
      protocol: TCP
      port: 9849
      targetPort: 9849
    - name: tcp-7848
      protocol: TCP
      port: 7848
      targetPort: 7848
  selector:
    app: nacos
---
# 修改以下数据库配置
apiVersion: v1
kind: ConfigMap
metadata:
  name: nacos-cm
  namespace: nacoscluster
data:
  oracle.db.name: "traffic_dev"
  oracle.host: "192.168.200.122"
  oracle.password: "Sjgj20241107"
  oracle.port: '1521'
  oracle.user: "zkdn_os_init"
  nacos-auth-enable: 'true'
  nacos-auth-identity-key: nacosPlus496
  nacos-auth-identity-value: nacosPlus628
  nacos-auth-token: TmFjb3NQbHVzMDEyMzQ1Njc4OTAxMjM0NTY3ODk5ODc2NTQzMjEwMDEyMzQ1Njc4OTk4NzY1NDMyMTAwMTIzNDU2Nzg5
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nacos
  namespace: nacoscluster
spec:
  serviceName: nacos-headless
  replicas: 3
  template:
    metadata:
      labels:
        app: nacos
      annotations:
        pod.alpha.kubernetes.io/initialized: "true"
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: "app"
                    operator: In
                    values:
                      - nacos
              topologyKey: "kubernetes.io/hostname"
      initContainers:
        - name: peer-finder-plugin-install
          image: '192.168.200.133:8080/sjgj/nacos/nacos-peer-finder-plugin:1.1'
          resources: {}
          volumeMounts:
            - name: pv-storage-elastic-master
              mountPath: /home/nacos/plugins/peer-finder
              subPath: peer-finder
      containers:
        - name: nacos
          imagePullPolicy: Always
          image: 192.168.200.133:8080/sjgj/nacos/nacos-server:v2.3.2
          resources:
            requests:
              memory: "2Gi"
              cpu: "500m"
          ports:
            - name: client-port
              containerPort: 8848
              protocol: TCP
            - name: client-rpc
              containerPort: 9848
              protocol: TCP
            - name: raft-rpc
              containerPort: 9849
              protocol: TCP
            - name: old-raft-rpc
              containerPort: 7848
              protocol: TCP
          env:
            - name: NACOS_REPLICAS
              value: "3"
            - name: ORACLE_SERVICE_HOST
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: oracle.host
            - name: ORACLE_SERVICE_DB_NAME
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: oracle.db.name
            - name: ORACLE_SERVICE_PORT
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: oracle.port
            - name: ORACLE_SERVICE_USER
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: oracle.user
            - name: ORACLE_SERVICE_PASSWORD
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: oracle.password
            - name: MODE
              value: "cluster"
            - name: SPRING_DATASOURCE_PLATFORM
              value: oracle
            - name: NACOS_APPLICATION_PORT
              value: "8848"
            - name: PREFER_HOST_MODE
              value: "hostname"
            - name: NACOS_SERVERS
              value: "nacos-0.nacos-headless.nacoscluster.svc.cluster.local:8848 nacos-1.nacos-headless.nacoscluster.svc.cluster.local:8848 nacos-2.nacos-headless.nacoscluster.svc.cluster.local:8848"
            - name: NACOS_AUTH_ENABLE
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: nacos-auth-enable
            - name: NACOS_AUTH_IDENTITY_KEY
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: nacos-auth-identity-key
            - name: NACOS_AUTH_IDENTITY_VALUE
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: nacos-auth-identity-value
            - name: NACOS_AUTH_TOKEN
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: nacos-auth-token
          volumeMounts:
            - name: pv-storage-elastic-master
              mountPath: /home/nacos/peer-finder
  volumeClaimTemplates:
  - metadata:
      name: pv-storage-elastic-master
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "local-storage2"
      resources:
        requests:
          storage: 100Gi
  selector:
    matchLabels:
      app: nacos

kubectl apply -f nacos-pvc.yaml

5.创建服务端口

bash 复制代码 
cat nacos-service.yaml

#特别强调一点，k8s的版本不同ingress配置的一些细节不同，具体可以看看官网，
#我展示的配置信息，只能保证1.23.1版本的k8s可以使用。
#nacos-k8s里也有ingress的配置，可以参考一下
#./nacos-k8s/deploy/nacos/nacos-no-pvc-ingress.yaml
apiVersion: v1
kind: Service
metadata:
  name: nacos
  namespace: nacoscluster
spec:
  ports:
    - name: tcp-8848
      protocol: TCP
      port: 8848
      targetPort: 8848
    - name: tcp-9848
      protocol: TCP
      port: 9848
      targetPort: 9848
    - name: tcp-9849
      protocol: TCP
      port: 9849
      targetPort: 9849
    - name: tcp-7848
      protocol: TCP
      port: 7848
      targetPort: 7848
  selector:
    app: nacos
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nacos
  namespace: nacoscluster
  labels:
    nacos: ingress-http
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - host: nacos.hdjsjb.com #自定义,ingress规定只能使用域名，没有的可以去修改host文件
    http:
      paths:
      - path: / #自定义,建议就这样，不然在springCloud服务注册时报错405
        pathType: Prefix #必须配置匹配策略
        backend:
          service: 
            name: nacos-headless
            port: 
              number: 8848

kubectl apply -f nacos-service.yaml

说明：可以在kuboard上查看


6.Nacos镜像内application.properties配置展示

bash 复制代码 
cat application.properties

# nacos-oracle-config
# spring
server.servlet.contextPath=${SERVER_SERVLET_CONTEXTPATH:/nacos}
server.contextPath=/nacos
server.port=${NACOS_APPLICATION_PORT:8848}
server.tomcat.accesslog.max-days=30
server.tomcat.accesslog.pattern=%h %l %u %t "%r" %s %b %D %{User-Agent}i %{Request-Source}i
server.tomcat.accesslog.enabled=${TOMCAT_ACCESSLOG_ENABLED:false}
server.error.include-message=ALWAYS
# default current work dir
server.tomcat.basedir=file:.
#*************** Config Module Related Configurations ***************#
### Deprecated configuration property, it is recommended to use `spring.sql.init.platform` replaced.
#spring.datasource.platform=${SPRING_DATASOURCE_PLATFORM:}
nacos.plugin.datasource.log.enabled=true

##### If use Oracle as datasource:
spring.sql.init.platform=${SPRING_DATASOURCE_PLATFORM:}
nacos.cmdb.dumpTaskInterval=3600
nacos.cmdb.eventTaskInterval=10
nacos.cmdb.labelTaskInterval=300
nacos.cmdb.loadDataAtStart=false
db.num=${ORACLE_DATABASE_NUM:1}
db.url.0=jdbc:oracle:thin:@//${ORACLE_SERVICE_HOST}:${ORACLE_SERVICE_PORT:1521}/${ORACLE_SERVICE_DB_NAME}
db.user.0=${ORACLE_SERVICE_USER}
db.password.0=${ORACLE_SERVICE_PASSWORD}
## DB connection pool settings
db.pool.config.connectionTimeout=${DB_POOL_CONNECTION_TIMEOUT:30000}
db.pool.config.validationTimeout=10000
db.pool.config.maximumPoolSize=20
db.pool.config.minimumIdle=2
db.pool.config.driver-class-name=oracle.jdbc.OracleDriver
db.testQuery=select 1 from dual
### The auth system to use, currently only 'nacos' and 'ldap' is supported:
nacos.core.auth.enabled=${NACOS_AUTH_ENABLE:false}
nacos.core.auth.system.type=${NACOS_AUTH_SYSTEM_TYPE:nacos}
### worked when nacos.core.auth.system.type=nacos
### The token expiration in seconds:
nacos.core.auth.plugin.nacos.token.expire.seconds=${NACOS_AUTH_TOKEN_EXPIRE_SECONDS:18000}
### The default token:
nacos.core.auth.plugin.nacos.token.secret.key=${NACOS_AUTH_TOKEN:}
### Turn on/off caching of auth information. By turning on this switch, the update of auth information would have a 15 seconds delay.
nacos.core.auth.caching.enabled=${NACOS_AUTH_CACHE_ENABLE:false}
nacos.core.auth.enable.userAgentAuthWhite=${NACOS_AUTH_USER_AGENT_AUTH_WHITE_ENABLE:false}
nacos.core.auth.server.identity.key=${NACOS_AUTH_IDENTITY_KEY:}
nacos.core.auth.server.identity.value=${NACOS_AUTH_IDENTITY_VALUE:}
## spring security config
### turn off security
nacos.security.ignore.urls=${NACOS_SECURITY_IGNORE_URLS:/,/error,/**/*.css,/**/*.js,/**/*.html,/**/*.map,/**/*.svg,/**/*.png,/**/*.ico,/console-fe/public/**,/v1/auth/**,/v1/console/health/**,/actuator/**,/v1/console/server/**}

# metrics for elastic search
management.metrics.export.elastic.enabled=false
management.metrics.export.influx.enabled=false
nacos.naming.distro.taskDispatchThreadCount=10
nacos.naming.distro.taskDispatchPeriod=200
nacos.naming.distro.batchSyncKeyCount=1000
nacos.naming.distro.initDataRatio=0.9
nacos.naming.distro.syncRetryDelay=5000
nacos.naming.data.warmup=true
nacos.console.ui.enabled=true
nacos.core.param.check.enabled=true

五、测试

用户名/密码：nacos/nacos

http://< IP >:8848/nacos/#/login



六、结束语

  到此k8s上Nacos集群部署Oracle作外部数据源使用讲解就完毕啦，如果有其它问题欢迎大家评论留言，如果觉得还可以，那么给作者点赞、收藏、关注吧！‌

相关推荐
运维开发故事2 天前
基于 Arthas 的多集群在线诊断系统设计与实现
kubernetes
Patrick_Wilson3 天前
从「改个端口」到 502：Next.js on k8s 的容器端口、Service 映射与 env 覆盖
docker·kubernetes·next.js
探索云原生4 天前
K8s 1.36 这个 GA 特性，把 initContainer 拉模型的 hack 干掉了
ai·云原生·kubernetes
云恒要逆袭4 天前
运行你的第一个Docker容器
后端·docker·容器
ClouGence5 天前
Oracle 数据同步为什么会出现数据不一致？长事务是常被忽略的原因
数据库·后端·oracle
Java之美5 天前
一次k8s升级引发的DevicePlugin注册失败
云原生·kubernetes
程序员老赵5 天前
10 分钟部署 OpenCode：Docker 一键安装，浏览器打开就能用 AI 写代码（附完整命令与排错）
docker·容器·ai编程
武子康8 天前
调查研究-183 Apple container：Mac 上用轻量 VM 跑 Linux 容器，Swift 会改写本地容器体验吗？
docker·容器·apple
ClouGence11 天前
Oracle CDC 架构优化：从主库直连到 DataGuard 备库同步
数据库·后端·oracle
热门推荐
012026年6月AI大模型全景报告：GPT-5.6、Claude Opus 4.8、Gemini 3.5，中美AI三足鼎立谁主沉浮？022026年6月AI行业全景：从百模大战到Agent元年，这30天发生了什么？03飞书长连接_事件订阅（接收消息，审批任务状态变更）04Trae国际版与国内版深度测评：AI原生IDE的双生花052026 年 AI 编程工具终极横评：Cursor vs Claude Code vs Copilot vs Windsurf06GitHub 镜像站点07【AI】2026 年具身智能模型和世界模型总结08Claude Code、Codex、Cursor三分天下：2026年AI编程Agent生态全景剖析092026年AI架构实战：彻底解决OpenAI接口超时与封号，Python调用GPT-5.2/Sora2企业级架构详解（附源码+压测报告）102026 AI 编程工具终极实战指南：Cursor vs Claude Code vs Copilot，开发者该怎么选？