K8S离线部署Nacos集群【Oracle作外部数据源】

cs阿坤dn2024-12-06 12:28

一、前言

由于公司的要求下要使Nacos集群以Oracle作为外部数据源，前期咱们已经阐述了如何在本地搭建（Nacos集群搭建【Oracle作外部数据源】），本次将带领大家在k8s上部署Nacos集群并以Oracle作为外部数据源。

二、软件包

说明：nacos-server-oracle.tar该镜像包经过本人dockerfile构建修改、加入了oracle相关变量、参数、以及支持oracle 11g的二次编译nacos-server.jar程序包。

三、环境

  • k8s1.23.1
  • nacos-server2.3.2
  • Oracle 11g RAC集群

四、部署

1.创建命名空间

bash 复制代码 
ls
cat 00-ns.yaml 

apiVersion: v1
kind: Namespace
metadata:
  name: nacoscluster
  labels:
    app: nacoscluster
    
kubectl apply -f 00-ns.yaml

2.创建存储类

bash 复制代码 
cat 00-sc.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: local-storage2
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

kubectl apply -f 00-sc.yaml

3.创建pv卷

bash 复制代码 
 cat pv.yaml 
apiVersion: v1
kind: PersistentVolume
metadata:
  name: local-storage-nacos-pv-0
  namespace: nacoscluster     # 空间名称
  labels:
    name: local-storage-nacos-pv-0
spec:
  capacity:
    storage: 100Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage2
  local:
    path: /srv/nacos_data #存储卷的实际位置，与nfs暴露的文件地址相同
  nodeAffinity:
    required:
      nodeSelectorTerms:        # 节点选择
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - node3            # 根据自己节点名称
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: local-storage-nacos-pv-1
  namespace: nacoscluster     # 空间名称
  labels:
    name: local-storage-nacos-pv-1
spec:
  capacity:
    storage: 100Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage2
  local:
    path: /srv/nacos_data #存储卷的实际位置，与nfs暴露的文件地址相同
  nodeAffinity:
    required:
      nodeSelectorTerms:        # 节点选择
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - node4            # 根据自己节点名称
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: local-storage-nacos-pv-2
  namespace: nacoscluster     # 空间名称
  labels:
    name: local-storage-nacos-pv-2
spec:
  capacity:
    storage: 100Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage2
  local:
    path: /srv/nacos_data #存储卷的实际位置，与nfs暴露的文件地址相同
  nodeAffinity:
    required:
      nodeSelectorTerms:        # 节点选择
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - node5            # 根据自己节点名称

kubectl apply -f  pv.yaml

注意：需要在对应节点(例：node3、node4、node5)创建该目录/srv/nacos_data

3.给相应节点打标签

bash 复制代码 
# 用于nacos容器绑定到指定节点（可选）
kubectl label nodes node3 app=nacos
kubectl label nodes node4 app=nacos
kubectl label nodes node5 app=nacos

4.创建nacos集群

bash 复制代码 
cat nacos-pvc.yaml

###使用自建数据库；使用Ingress发布配置后台###
---
apiVersion: v1
kind: Service
metadata:
  name: nacos-headless
  namespace: nacoscluster
  labels:
    app: nacos-headless
spec:
  type: ClusterIP
  clusterIP: None
  ports:
    - name: tcp-8848
      protocol: TCP
      port: 8848
      targetPort: 8848
    - name: tcp-9848
      protocol: TCP
      port: 9848
      targetPort: 9848
    - name: tcp-9849
      protocol: TCP
      port: 9849
      targetPort: 9849
    - name: tcp-7848
      protocol: TCP
      port: 7848
      targetPort: 7848
  selector:
    app: nacos
---
# 修改以下数据库配置
apiVersion: v1
kind: ConfigMap
metadata:
  name: nacos-cm
  namespace: nacoscluster
data:
  oracle.db.name: "traffic_dev"
  oracle.host: "192.168.200.122"
  oracle.password: "Sjgj20241107"
  oracle.port: '1521'
  oracle.user: "zkdn_os_init"
  nacos-auth-enable: 'true'
  nacos-auth-identity-key: nacosPlus496
  nacos-auth-identity-value: nacosPlus628
  nacos-auth-token: TmFjb3NQbHVzMDEyMzQ1Njc4OTAxMjM0NTY3ODk5ODc2NTQzMjEwMDEyMzQ1Njc4OTk4NzY1NDMyMTAwMTIzNDU2Nzg5
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nacos
  namespace: nacoscluster
spec:
  serviceName: nacos-headless
  replicas: 3
  template:
    metadata:
      labels:
        app: nacos
      annotations:
        pod.alpha.kubernetes.io/initialized: "true"
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: "app"
                    operator: In
                    values:
                      - nacos
              topologyKey: "kubernetes.io/hostname"
      initContainers:
        - name: peer-finder-plugin-install
          image: '192.168.200.133:8080/sjgj/nacos/nacos-peer-finder-plugin:1.1'
          resources: {}
          volumeMounts:
            - name: pv-storage-elastic-master
              mountPath: /home/nacos/plugins/peer-finder
              subPath: peer-finder
      containers:
        - name: nacos
          imagePullPolicy: Always
          image: 192.168.200.133:8080/sjgj/nacos/nacos-server:v2.3.2
          resources:
            requests:
              memory: "2Gi"
              cpu: "500m"
          ports:
            - name: client-port
              containerPort: 8848
              protocol: TCP
            - name: client-rpc
              containerPort: 9848
              protocol: TCP
            - name: raft-rpc
              containerPort: 9849
              protocol: TCP
            - name: old-raft-rpc
              containerPort: 7848
              protocol: TCP
          env:
            - name: NACOS_REPLICAS
              value: "3"
            - name: ORACLE_SERVICE_HOST
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: oracle.host
            - name: ORACLE_SERVICE_DB_NAME
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: oracle.db.name
            - name: ORACLE_SERVICE_PORT
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: oracle.port
            - name: ORACLE_SERVICE_USER
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: oracle.user
            - name: ORACLE_SERVICE_PASSWORD
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: oracle.password
            - name: MODE
              value: "cluster"
            - name: SPRING_DATASOURCE_PLATFORM
              value: oracle
            - name: NACOS_APPLICATION_PORT
              value: "8848"
            - name: PREFER_HOST_MODE
              value: "hostname"
            - name: NACOS_SERVERS
              value: "nacos-0.nacos-headless.nacoscluster.svc.cluster.local:8848 nacos-1.nacos-headless.nacoscluster.svc.cluster.local:8848 nacos-2.nacos-headless.nacoscluster.svc.cluster.local:8848"
            - name: NACOS_AUTH_ENABLE
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: nacos-auth-enable
            - name: NACOS_AUTH_IDENTITY_KEY
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: nacos-auth-identity-key
            - name: NACOS_AUTH_IDENTITY_VALUE
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: nacos-auth-identity-value
            - name: NACOS_AUTH_TOKEN
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: nacos-auth-token
          volumeMounts:
            - name: pv-storage-elastic-master
              mountPath: /home/nacos/peer-finder
  volumeClaimTemplates:
  - metadata:
      name: pv-storage-elastic-master
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "local-storage2"
      resources:
        requests:
          storage: 100Gi
  selector:
    matchLabels:
      app: nacos

kubectl apply -f nacos-pvc.yaml

5.创建服务端口

bash 复制代码 
cat nacos-service.yaml

#特别强调一点，k8s的版本不同ingress配置的一些细节不同，具体可以看看官网，
#我展示的配置信息，只能保证1.23.1版本的k8s可以使用。
#nacos-k8s里也有ingress的配置，可以参考一下
#./nacos-k8s/deploy/nacos/nacos-no-pvc-ingress.yaml
apiVersion: v1
kind: Service
metadata:
  name: nacos
  namespace: nacoscluster
spec:
  ports:
    - name: tcp-8848
      protocol: TCP
      port: 8848
      targetPort: 8848
    - name: tcp-9848
      protocol: TCP
      port: 9848
      targetPort: 9848
    - name: tcp-9849
      protocol: TCP
      port: 9849
      targetPort: 9849
    - name: tcp-7848
      protocol: TCP
      port: 7848
      targetPort: 7848
  selector:
    app: nacos
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nacos
  namespace: nacoscluster
  labels:
    nacos: ingress-http
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - host: nacos.hdjsjb.com #自定义,ingress规定只能使用域名，没有的可以去修改host文件
    http:
      paths:
      - path: / #自定义,建议就这样，不然在springCloud服务注册时报错405
        pathType: Prefix #必须配置匹配策略
        backend:
          service: 
            name: nacos-headless
            port: 
              number: 8848

kubectl apply -f nacos-service.yaml

说明：可以在kuboard上查看


6.Nacos镜像内application.properties配置展示

bash 复制代码 
cat application.properties

# nacos-oracle-config
# spring
server.servlet.contextPath=${SERVER_SERVLET_CONTEXTPATH:/nacos}
server.contextPath=/nacos
server.port=${NACOS_APPLICATION_PORT:8848}
server.tomcat.accesslog.max-days=30
server.tomcat.accesslog.pattern=%h %l %u %t "%r" %s %b %D %{User-Agent}i %{Request-Source}i
server.tomcat.accesslog.enabled=${TOMCAT_ACCESSLOG_ENABLED:false}
server.error.include-message=ALWAYS
# default current work dir
server.tomcat.basedir=file:.
#*************** Config Module Related Configurations ***************#
### Deprecated configuration property, it is recommended to use `spring.sql.init.platform` replaced.
#spring.datasource.platform=${SPRING_DATASOURCE_PLATFORM:}
nacos.plugin.datasource.log.enabled=true

##### If use Oracle as datasource:
spring.sql.init.platform=${SPRING_DATASOURCE_PLATFORM:}
nacos.cmdb.dumpTaskInterval=3600
nacos.cmdb.eventTaskInterval=10
nacos.cmdb.labelTaskInterval=300
nacos.cmdb.loadDataAtStart=false
db.num=${ORACLE_DATABASE_NUM:1}
db.url.0=jdbc:oracle:thin:@//${ORACLE_SERVICE_HOST}:${ORACLE_SERVICE_PORT:1521}/${ORACLE_SERVICE_DB_NAME}
db.user.0=${ORACLE_SERVICE_USER}
db.password.0=${ORACLE_SERVICE_PASSWORD}
## DB connection pool settings
db.pool.config.connectionTimeout=${DB_POOL_CONNECTION_TIMEOUT:30000}
db.pool.config.validationTimeout=10000
db.pool.config.maximumPoolSize=20
db.pool.config.minimumIdle=2
db.pool.config.driver-class-name=oracle.jdbc.OracleDriver
db.testQuery=select 1 from dual
### The auth system to use, currently only 'nacos' and 'ldap' is supported:
nacos.core.auth.enabled=${NACOS_AUTH_ENABLE:false}
nacos.core.auth.system.type=${NACOS_AUTH_SYSTEM_TYPE:nacos}
### worked when nacos.core.auth.system.type=nacos
### The token expiration in seconds:
nacos.core.auth.plugin.nacos.token.expire.seconds=${NACOS_AUTH_TOKEN_EXPIRE_SECONDS:18000}
### The default token:
nacos.core.auth.plugin.nacos.token.secret.key=${NACOS_AUTH_TOKEN:}
### Turn on/off caching of auth information. By turning on this switch, the update of auth information would have a 15 seconds delay.
nacos.core.auth.caching.enabled=${NACOS_AUTH_CACHE_ENABLE:false}
nacos.core.auth.enable.userAgentAuthWhite=${NACOS_AUTH_USER_AGENT_AUTH_WHITE_ENABLE:false}
nacos.core.auth.server.identity.key=${NACOS_AUTH_IDENTITY_KEY:}
nacos.core.auth.server.identity.value=${NACOS_AUTH_IDENTITY_VALUE:}
## spring security config
### turn off security
nacos.security.ignore.urls=${NACOS_SECURITY_IGNORE_URLS:/,/error,/**/*.css,/**/*.js,/**/*.html,/**/*.map,/**/*.svg,/**/*.png,/**/*.ico,/console-fe/public/**,/v1/auth/**,/v1/console/health/**,/actuator/**,/v1/console/server/**}

# metrics for elastic search
management.metrics.export.elastic.enabled=false
management.metrics.export.influx.enabled=false
nacos.naming.distro.taskDispatchThreadCount=10
nacos.naming.distro.taskDispatchPeriod=200
nacos.naming.distro.batchSyncKeyCount=1000
nacos.naming.distro.initDataRatio=0.9
nacos.naming.distro.syncRetryDelay=5000
nacos.naming.data.warmup=true
nacos.console.ui.enabled=true
nacos.core.param.check.enabled=true

五、测试

用户名/密码：nacos/nacos

http://< IP >:8848/nacos/#/login



六、结束语

到此k8s上Nacos集群部署Oracle作外部数据源使用讲解就完毕啦，如果有其它问题欢迎大家评论留言，如果觉得还可以，那么给作者点赞、收藏、关注吧！‌

相关推荐
Andy杨17 分钟前
20250707-4-Kubernetes 集群部署、配置和验证-K8s基本资源概念初_笔记
笔记·容器·kubernetes
容器魔方3 小时前
中选名单出炉|18位学生入选开源之夏KubeEdge课题，欢迎加入！
云原生·容器·云计算
发仔1234 小时前
Oracle与MySQL核心差异对比
mysql·oracle
love530love5 小时前
Docker 稳定运行与存储优化全攻略（含可视化指南）
运维·人工智能·windows·docker·容器
椰椰椰耶5 小时前
【MyBatis】XML实现，配置方法和增、删、改、查
xml·oracle·mybatis
人生匆匆5 小时前
docker进入启动失败的容器
运维·docker·容器
赵渝强老师6 小时前
【赵渝强老师】Oracle RMAN的目录数据库
数据库·oracle
暖暖木头6 小时前
Oracle注释详解
数据库·oracle
退役小学生呀7 小时前
十、K8s集群资源合理化分配
linux·云原生·容器·kubernetes·k8s
GJCTYU8 小时前
spring中@Transactional注解和事务的实战理解附代码
数据库·spring boot·后端·spring·oracle·mybatis
热门推荐
01Java学习第十五部分——MyBatis02集群聊天服务器---MySQL数据库的建立03Coze扣子平台完整体验和实践（附国内和国际版对比）04基于odoo17的设计模式详解---装饰模式05使用Ruby接入实时行情API教程06扣子（coze）实战|我用扣子搭建了一个自动分析小红薯笔记内容的AI应用|详细步骤拆解07Everything文件检索工具 几秒检索几百G的文件08基于odoo17的设计模式详解---单例模式09DeepSeek各版本说明与优缺点分析10【无标题】