K8S离线部署Nacos集群【Oracle作外部数据源】

cs阿坤dn2024-12-06 12:28

一、前言

由于公司的要求下要使Nacos集群以Oracle作为外部数据源，前期咱们已经阐述了如何在本地搭建（Nacos集群搭建【Oracle作外部数据源】），本次将带领大家在k8s上部署Nacos集群并以Oracle作为外部数据源。

二、软件包

说明：nacos-server-oracle.tar该镜像包经过本人dockerfile构建修改、加入了oracle相关变量、参数、以及支持oracle 11g的二次编译nacos-server.jar程序包。

三、环境

  • k8s1.23.1
  • nacos-server2.3.2
  • Oracle 11g RAC集群

四、部署

1.创建命名空间

bash 复制代码 
ls
cat 00-ns.yaml 

apiVersion: v1
kind: Namespace
metadata:
  name: nacoscluster
  labels:
    app: nacoscluster
    
kubectl apply -f 00-ns.yaml

2.创建存储类

bash 复制代码 
cat 00-sc.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: local-storage2
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

kubectl apply -f 00-sc.yaml

3.创建pv卷

bash 复制代码 
 cat pv.yaml 
apiVersion: v1
kind: PersistentVolume
metadata:
  name: local-storage-nacos-pv-0
  namespace: nacoscluster     # 空间名称
  labels:
    name: local-storage-nacos-pv-0
spec:
  capacity:
    storage: 100Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage2
  local:
    path: /srv/nacos_data #存储卷的实际位置，与nfs暴露的文件地址相同
  nodeAffinity:
    required:
      nodeSelectorTerms:        # 节点选择
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - node3            # 根据自己节点名称
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: local-storage-nacos-pv-1
  namespace: nacoscluster     # 空间名称
  labels:
    name: local-storage-nacos-pv-1
spec:
  capacity:
    storage: 100Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage2
  local:
    path: /srv/nacos_data #存储卷的实际位置，与nfs暴露的文件地址相同
  nodeAffinity:
    required:
      nodeSelectorTerms:        # 节点选择
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - node4            # 根据自己节点名称
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: local-storage-nacos-pv-2
  namespace: nacoscluster     # 空间名称
  labels:
    name: local-storage-nacos-pv-2
spec:
  capacity:
    storage: 100Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage2
  local:
    path: /srv/nacos_data #存储卷的实际位置，与nfs暴露的文件地址相同
  nodeAffinity:
    required:
      nodeSelectorTerms:        # 节点选择
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - node5            # 根据自己节点名称

kubectl apply -f  pv.yaml

注意：需要在对应节点(例：node3、node4、node5)创建该目录/srv/nacos_data

3.给相应节点打标签

bash 复制代码 
# 用于nacos容器绑定到指定节点（可选）
kubectl label nodes node3 app=nacos
kubectl label nodes node4 app=nacos
kubectl label nodes node5 app=nacos

4.创建nacos集群

bash 复制代码 
cat nacos-pvc.yaml

###使用自建数据库；使用Ingress发布配置后台###
---
apiVersion: v1
kind: Service
metadata:
  name: nacos-headless
  namespace: nacoscluster
  labels:
    app: nacos-headless
spec:
  type: ClusterIP
  clusterIP: None
  ports:
    - name: tcp-8848
      protocol: TCP
      port: 8848
      targetPort: 8848
    - name: tcp-9848
      protocol: TCP
      port: 9848
      targetPort: 9848
    - name: tcp-9849
      protocol: TCP
      port: 9849
      targetPort: 9849
    - name: tcp-7848
      protocol: TCP
      port: 7848
      targetPort: 7848
  selector:
    app: nacos
---
# 修改以下数据库配置
apiVersion: v1
kind: ConfigMap
metadata:
  name: nacos-cm
  namespace: nacoscluster
data:
  oracle.db.name: "traffic_dev"
  oracle.host: "192.168.200.122"
  oracle.password: "Sjgj20241107"
  oracle.port: '1521'
  oracle.user: "zkdn_os_init"
  nacos-auth-enable: 'true'
  nacos-auth-identity-key: nacosPlus496
  nacos-auth-identity-value: nacosPlus628
  nacos-auth-token: TmFjb3NQbHVzMDEyMzQ1Njc4OTAxMjM0NTY3ODk5ODc2NTQzMjEwMDEyMzQ1Njc4OTk4NzY1NDMyMTAwMTIzNDU2Nzg5
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nacos
  namespace: nacoscluster
spec:
  serviceName: nacos-headless
  replicas: 3
  template:
    metadata:
      labels:
        app: nacos
      annotations:
        pod.alpha.kubernetes.io/initialized: "true"
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: "app"
                    operator: In
                    values:
                      - nacos
              topologyKey: "kubernetes.io/hostname"
      initContainers:
        - name: peer-finder-plugin-install
          image: '192.168.200.133:8080/sjgj/nacos/nacos-peer-finder-plugin:1.1'
          resources: {}
          volumeMounts:
            - name: pv-storage-elastic-master
              mountPath: /home/nacos/plugins/peer-finder
              subPath: peer-finder
      containers:
        - name: nacos
          imagePullPolicy: Always
          image: 192.168.200.133:8080/sjgj/nacos/nacos-server:v2.3.2
          resources:
            requests:
              memory: "2Gi"
              cpu: "500m"
          ports:
            - name: client-port
              containerPort: 8848
              protocol: TCP
            - name: client-rpc
              containerPort: 9848
              protocol: TCP
            - name: raft-rpc
              containerPort: 9849
              protocol: TCP
            - name: old-raft-rpc
              containerPort: 7848
              protocol: TCP
          env:
            - name: NACOS_REPLICAS
              value: "3"
            - name: ORACLE_SERVICE_HOST
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: oracle.host
            - name: ORACLE_SERVICE_DB_NAME
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: oracle.db.name
            - name: ORACLE_SERVICE_PORT
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: oracle.port
            - name: ORACLE_SERVICE_USER
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: oracle.user
            - name: ORACLE_SERVICE_PASSWORD
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: oracle.password
            - name: MODE
              value: "cluster"
            - name: SPRING_DATASOURCE_PLATFORM
              value: oracle
            - name: NACOS_APPLICATION_PORT
              value: "8848"
            - name: PREFER_HOST_MODE
              value: "hostname"
            - name: NACOS_SERVERS
              value: "nacos-0.nacos-headless.nacoscluster.svc.cluster.local:8848 nacos-1.nacos-headless.nacoscluster.svc.cluster.local:8848 nacos-2.nacos-headless.nacoscluster.svc.cluster.local:8848"
            - name: NACOS_AUTH_ENABLE
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: nacos-auth-enable
            - name: NACOS_AUTH_IDENTITY_KEY
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: nacos-auth-identity-key
            - name: NACOS_AUTH_IDENTITY_VALUE
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: nacos-auth-identity-value
            - name: NACOS_AUTH_TOKEN
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: nacos-auth-token
          volumeMounts:
            - name: pv-storage-elastic-master
              mountPath: /home/nacos/peer-finder
  volumeClaimTemplates:
  - metadata:
      name: pv-storage-elastic-master
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "local-storage2"
      resources:
        requests:
          storage: 100Gi
  selector:
    matchLabels:
      app: nacos

kubectl apply -f nacos-pvc.yaml

5.创建服务端口

bash 复制代码 
cat nacos-service.yaml

#特别强调一点，k8s的版本不同ingress配置的一些细节不同，具体可以看看官网，
#我展示的配置信息，只能保证1.23.1版本的k8s可以使用。
#nacos-k8s里也有ingress的配置，可以参考一下
#./nacos-k8s/deploy/nacos/nacos-no-pvc-ingress.yaml
apiVersion: v1
kind: Service
metadata:
  name: nacos
  namespace: nacoscluster
spec:
  ports:
    - name: tcp-8848
      protocol: TCP
      port: 8848
      targetPort: 8848
    - name: tcp-9848
      protocol: TCP
      port: 9848
      targetPort: 9848
    - name: tcp-9849
      protocol: TCP
      port: 9849
      targetPort: 9849
    - name: tcp-7848
      protocol: TCP
      port: 7848
      targetPort: 7848
  selector:
    app: nacos
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nacos
  namespace: nacoscluster
  labels:
    nacos: ingress-http
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - host: nacos.hdjsjb.com #自定义,ingress规定只能使用域名，没有的可以去修改host文件
    http:
      paths:
      - path: / #自定义,建议就这样，不然在springCloud服务注册时报错405
        pathType: Prefix #必须配置匹配策略
        backend:
          service: 
            name: nacos-headless
            port: 
              number: 8848

kubectl apply -f nacos-service.yaml

说明：可以在kuboard上查看


6.Nacos镜像内application.properties配置展示

bash 复制代码 
cat application.properties

# nacos-oracle-config
# spring
server.servlet.contextPath=${SERVER_SERVLET_CONTEXTPATH:/nacos}
server.contextPath=/nacos
server.port=${NACOS_APPLICATION_PORT:8848}
server.tomcat.accesslog.max-days=30
server.tomcat.accesslog.pattern=%h %l %u %t "%r" %s %b %D %{User-Agent}i %{Request-Source}i
server.tomcat.accesslog.enabled=${TOMCAT_ACCESSLOG_ENABLED:false}
server.error.include-message=ALWAYS
# default current work dir
server.tomcat.basedir=file:.
#*************** Config Module Related Configurations ***************#
### Deprecated configuration property, it is recommended to use `spring.sql.init.platform` replaced.
#spring.datasource.platform=${SPRING_DATASOURCE_PLATFORM:}
nacos.plugin.datasource.log.enabled=true

##### If use Oracle as datasource:
spring.sql.init.platform=${SPRING_DATASOURCE_PLATFORM:}
nacos.cmdb.dumpTaskInterval=3600
nacos.cmdb.eventTaskInterval=10
nacos.cmdb.labelTaskInterval=300
nacos.cmdb.loadDataAtStart=false
db.num=${ORACLE_DATABASE_NUM:1}
db.url.0=jdbc:oracle:thin:@//${ORACLE_SERVICE_HOST}:${ORACLE_SERVICE_PORT:1521}/${ORACLE_SERVICE_DB_NAME}
db.user.0=${ORACLE_SERVICE_USER}
db.password.0=${ORACLE_SERVICE_PASSWORD}
## DB connection pool settings
db.pool.config.connectionTimeout=${DB_POOL_CONNECTION_TIMEOUT:30000}
db.pool.config.validationTimeout=10000
db.pool.config.maximumPoolSize=20
db.pool.config.minimumIdle=2
db.pool.config.driver-class-name=oracle.jdbc.OracleDriver
db.testQuery=select 1 from dual
### The auth system to use, currently only 'nacos' and 'ldap' is supported:
nacos.core.auth.enabled=${NACOS_AUTH_ENABLE:false}
nacos.core.auth.system.type=${NACOS_AUTH_SYSTEM_TYPE:nacos}
### worked when nacos.core.auth.system.type=nacos
### The token expiration in seconds:
nacos.core.auth.plugin.nacos.token.expire.seconds=${NACOS_AUTH_TOKEN_EXPIRE_SECONDS:18000}
### The default token:
nacos.core.auth.plugin.nacos.token.secret.key=${NACOS_AUTH_TOKEN:}
### Turn on/off caching of auth information. By turning on this switch, the update of auth information would have a 15 seconds delay.
nacos.core.auth.caching.enabled=${NACOS_AUTH_CACHE_ENABLE:false}
nacos.core.auth.enable.userAgentAuthWhite=${NACOS_AUTH_USER_AGENT_AUTH_WHITE_ENABLE:false}
nacos.core.auth.server.identity.key=${NACOS_AUTH_IDENTITY_KEY:}
nacos.core.auth.server.identity.value=${NACOS_AUTH_IDENTITY_VALUE:}
## spring security config
### turn off security
nacos.security.ignore.urls=${NACOS_SECURITY_IGNORE_URLS:/,/error,/**/*.css,/**/*.js,/**/*.html,/**/*.map,/**/*.svg,/**/*.png,/**/*.ico,/console-fe/public/**,/v1/auth/**,/v1/console/health/**,/actuator/**,/v1/console/server/**}

# metrics for elastic search
management.metrics.export.elastic.enabled=false
management.metrics.export.influx.enabled=false
nacos.naming.distro.taskDispatchThreadCount=10
nacos.naming.distro.taskDispatchPeriod=200
nacos.naming.distro.batchSyncKeyCount=1000
nacos.naming.distro.initDataRatio=0.9
nacos.naming.distro.syncRetryDelay=5000
nacos.naming.data.warmup=true
nacos.console.ui.enabled=true
nacos.core.param.check.enabled=true

五、测试

用户名/密码：nacos/nacos

http://< IP >:8848/nacos/#/login



六、结束语

到此k8s上Nacos集群部署Oracle作外部数据源使用讲解就完毕啦，如果有其它问题欢迎大家评论留言，如果觉得还可以，那么给作者点赞、收藏、关注吧！‌

相关推荐
焱焱枫6 小时前
Oracle获取执行计划之10046 技术详解
数据库·oracle
cui_hao_nan7 小时前
Docker后端部署
运维·docker·容器
小张是铁粉8 小时前
docker在Linux的安装遇到的问题
linux·docker·容器
没有名字的小羊10 小时前
8.Docker镜像讲解
运维·docker·容器·tomcat
木鱼时刻11 小时前
容器与 Kubernetes 基本概念与架构
容器·架构·kubernetes
做一个AC梦13 小时前
Docker安装失败：Docker Desktop installation failed
运维·docker·容器
Shan120513 小时前
浅谈Docker Kicks in的应用
运维·docker·容器
Li&&Tao13 小时前
docker 常用命令
docker·容器·eureka
chuanauc21 小时前
Kubernets K8s 学习
java·学习·kubernetes
小张是铁粉21 小时前
docker学习二天之镜像操作与容器操作
学习·docker·容器
热门推荐
01GPU 进阶笔记（二）：华为昇腾 910B GPU02Word粘贴时出现“运行时错误53,文件未找到：MathPage.WLL“的解决方案03Coze扣子平台完整体验和实践（附国内和国际版对比）04MIUI显示/隐藏5G开关的方法，信号弱时开启手机Wifi通话方法05扣子空间的使用教程与大模型技术思考06扣子（coze）实战|我用扣子搭建了一个自动分析小红薯笔记内容的AI应用|详细步骤拆解07集群聊天服务器---MySQL数据库的建立08Coze实战第13讲：飞书多维表格读取+豆包生图模型，轻松批量生成短剧封面09DeepSeek各版本说明与优缺点分析10使用Ruby接入实时行情API教程