es+kibana---集群部署

Hfc.2025-04-30 15:53

其实一般es要跑3个节点的，这样才能做高可用，处理并发大，但是我这里只是一个pod

mkdir -p /stroe/data/es

es搭建：

#【拉取镜像】

#docker pull elasticsearch:6.8.7

#docker pull busybox:1.28

【导入镜像】

docker load -i es.tar

【创建命名空间】

kubectl create ns middle-ware

【创建es的资源】

vim test-es.yaml

复制代码 
---
# ConfigMap for Elasticsearch configuration
apiVersion: v1
kind: ConfigMap
metadata:
  name: es-config
  namespace: middle-ware
data:
  elasticsearch.yml: |
    cluster.name: my-es-cluster
    node.name: ${HOSTNAME}
    network.host: 0.0.0.0
    discovery.type: single-node
    path.data: /usr/share/elasticsearch/data
    path.logs: /usr/share/elasticsearch/logs
    bootstrap.memory_lock: false
    http.port: 9200
    transport.port: 9300
    
  jvm.options: |
    -Xms512m
    -Xmx512m
    -XX:+UseG1GC
    -XX:G1HeapRegionSize=4m
    -XX:MaxGCPauseMillis=50
    -XX:+PrintGCDetails
    -XX:+HeapDumpOnOutOfMemoryError
    -Xlog:gc*:file=/usr/share/elasticsearch/logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m

---
# PersistentVolume (使用 hostPath 本地存储)
apiVersion: v1
kind: PersistentVolume
metadata:
  name: es-pv
spec:
  capacity:
    storage: 1Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: manual
  hostPath:
    path: /data/elasticsearch
    type: DirectoryOrCreate

---
# PersistentVolumeClaim
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: es-pvc
  namespace: middle-ware
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

---
# StatefulSet
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: elasticsearch
  namespace: middle-ware
spec:
  serviceName: elasticsearch
  replicas: 1
  selector:
    matchLabels:
      app: elasticsearch
  template:
    metadata:
      labels:
        app: elasticsearch
    spec:
      tolerations:  #放在主节点上，需要配置污点容忍
      - key: "node-role.kubernetes.io/control-plane"
        operator: "Exists"
        effect: "NoSchedule"
      initContainers:
      - name: volume-permissions
        image: busybox:1.28
        command: ["sh", "-c", "chown -R 1000:0 /usr/share/elasticsearch/data /usr/share/elasticsearch/logs"]
        volumeMounts:
        - name: es-storage
          mountPath: /usr/share/elasticsearch/data
          subPath: data
        - name: es-storage
          mountPath: /usr/share/elasticsearch/logs
          subPath: logs
      containers:
      - name: elasticsearch
        image: elasticsearch:6.8.7
        imagePullPolicy: IfNotPresent
        env:
        - name: ES_JAVA_OPTS
          value: "-Xms512m -Xmx512m"
        - name: discovery.type
          value: single-node
        - name: TAKE_FILE_OWNERSHIP
          value: "true"
        ports:
        - containerPort: 9200
          name: http
        - containerPort: 9300
          name: transport
        volumeMounts:
        - name: es-config
          mountPath: /usr/share/elasticsearch/config/elasticsearch.yml
          subPath: elasticsearch.yml
        - name: es-config
          mountPath: /usr/share/elasticsearch/config/jvm.options
          subPath: jvm.options
        - name: es-storage
          mountPath: /usr/share/elasticsearch/data
          subPath: data
        - name: es-storage
          mountPath: /usr/share/elasticsearch/logs
          subPath: logs
        - name: es-storage
          mountPath: /usr/share/elasticsearch/plugins
          subPath: plugins
      volumes:
      - name: es-config
        configMap:
          name: es-config
          defaultMode: 0644
      - name: es-storage
        persistentVolumeClaim:
          claimName: es-pvc

---
# Service
apiVersion: v1
kind: Service
metadata:
  name: elasticsearch
  namespace: middle-ware
spec:
  selector:
    app: elasticsearch
  ports:
  - port: 9200
    name: http
    targetPort: 9200
  - port: 9300
    name: transport
    targetPort: 9300
  type: NodePort

vim es.yml 【这个是无https的，先跑起来生成证书放到宿主机】

复制代码 
---
# ConfigMap for Elasticsearch configuration
apiVersion: v1
kind: ConfigMap
metadata:
  name: es-config
  namespace: middle-ware-rp
data:
  elasticsearch.yml: |
    cluster.name: my-es-cluster
    node.name: ${HOSTNAME}
    network.host: 0.0.0.0
    discovery.type: single-node
    path.data: /usr/share/elasticsearch/data
    path.logs: /usr/share/elasticsearch/logs
    bootstrap.memory_lock: false
    http.port: 9200
    transport.port: 9300
    xpack.security.enabled: true
    xpack.security.transport.ssl.enabled: true
    xpack.security.http.ssl.enabled: false  # 先禁用 HTTP SSL 简化配置

    
  jvm.options: |
    -Xms1G
    -Xmx2G
    -XX:+UseG1GC
    -XX:G1HeapRegionSize=4m
    -XX:MaxGCPauseMillis=50
    -XX:+PrintGCDetails
    -XX:+HeapDumpOnOutOfMemoryError
    -Xlog:gc*:file=/usr/share/elasticsearch/logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m

---
# PersistentVolume (使用 hostPath 本地存储)
apiVersion: v1
kind: PersistentVolume
metadata:
  name: es-pv
spec:
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: manual
  hostPath:
    path: /store/data/es
    type: DirectoryOrCreate

---
# PersistentVolumeClaim
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: es-pvc
  namespace: middle-ware-rp
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

---
# StatefulSet
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: elasticsearch
  namespace: middle-ware-rp
spec:
  serviceName: elasticsearch
  replicas: 1
  selector:
    matchLabels:
      app: elasticsearch
  template:
    metadata:
      labels:
        app: elasticsearch
    spec:
      tolerations:  #放在主节点上，需要配置污点容忍
      - key: "node-role.kubernetes.io/control-plane"
        operator: "Exists"
        effect: "NoSchedule"
      nodeName: node2
      initContainers:
      - name: volume-permissions
        image: busybox:1.28
        command: ["sh", "-c", "chown -R 1000:0 /usr/share/elasticsearch/data /usr/share/elasticsearch/logs"]
        volumeMounts:
        - name: es-storage
          mountPath: /usr/share/elasticsearch/data
          subPath: data
        - name: es-storage
          mountPath: /usr/share/elasticsearch/logs
          subPath: logs
      containers:
      - name: elasticsearch
        image: elasticsearch:6.8.7
        imagePullPolicy: IfNotPresent
        env:
        - name: ES_JAVA_OPTS
          value: "-Xms1G -Xmx2G"
        - name: discovery.type
          value: single-node
        - name: TAKE_FILE_OWNERSHIP
          value: "true"
        ports:
        - containerPort: 9200
          name: http
        - containerPort: 9300
          name: transport
        volumeMounts:
        - name: es-config
          mountPath: /usr/share/elasticsearch/config/elasticsearch.yml
          subPath: elasticsearch.yml
        - name: es-config
          mountPath: /usr/share/elasticsearch/config/jvm.options
          subPath: jvm.options
        - name: es-storage
          mountPath: /usr/share/elasticsearch/data
          subPath: data
        - name: es-storage
          mountPath: /usr/share/elasticsearch/logs
          subPath: logs
        - name: es-storage
          mountPath: /usr/share/elasticsearch/plugins
          subPath: plugins
      volumes:
      - name: es-config
        configMap:
          name: es-config
          defaultMode: 0644
      - name: es-storage
        persistentVolumeClaim:
          claimName: es-pvc

---
# Service
apiVersion: v1
kind: Service
metadata:
  name: elasticsearch
  namespace: middle-ware-rp
spec:
  selector:
    app: elasticsearch
  ports:
  - port: 9200
    name: http
    targetPort: 9200
    nodePort: 30001
  - port: 9300
    name: transport
    targetPort: 9300
    nodePort: 30002
  type: NodePort

=====================【es1.yaml+https的url优化】==============================

证书生成的方式：

kubectl exec -it elasticsearch-0 -n middle-ware -- /bin/bash

./bin/elasticsearch-certutil ca #回车回车生成证书

./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 #回车回车生成证书

#！！拷贝到宿主机！！，这个ca证书就是开启es的https用来连接

复制代码 
证书认证：首先让你的es跑起来，然后在配置中应用es1.yaml优化配置文件
  elasticsearch.yml: |
    cluster.name: my-es-cluster
    node.name: ${HOSTNAME}
    network.host: 0.0.0.0
    discovery.type: single-node
    path.data: /usr/share/elasticsearch/data
    path.logs: /usr/share/elasticsearch/logs
    bootstrap.memory_lock: false
    http.port: 9200
    transport.port: 9300
    xpack.security.enabled: true
    xpack.security.transport.ssl.enabled: true
    xpack.security.http.ssl.enabled: false  # false先禁用 HTTP SSL 简化配置，无证书，有证书再更改为true再加入下面配置
    #有了证书之后加入一下配置，并且开启httpssl认证
    xpack.security.transport.ssl.keystore.type: PKCS12
    xpack.security.transport.ssl.verification_mode: certificate
    xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
    xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
    xpack.security.transport.ssl.truststore.type: PKCS12
    xpack.security.audit.enabled: true
    xpack.security.http.ssl.keystore.path: /usr/share/elasticsearch/config/elastic-certificates.p12
    xpack.security.http.ssl.truststore.path: /usr/share/elasticsearch/config/elastic-certificates.p12

#证书生成后，应用ess1.yaml文件，并且先生成secret资源
ls 
[root@150m01 ~/kind]# ll
-rw-r--r-- 1 root root        3443 4月  25 09:32 elastic-certificates.p12
-rw-r--r-- 1 root root        2527 4月  25 09:33 elastic-stack-ca.p12
#生成secret资源
kubectl -n middle-ware create secret generic es-cert --from-file=elastic-certificates.p12

#引用secret资源主要增加：
sts.es.spec.template.spec.containers.volumeMounts
        - name: es-cert
          mountPath: /usr/share/elasticsearch/config/elastic-certificates.p12
          subPath: elastic-certificates.p12

sts.es.spec.template.spec.volumes
      - name: es-cert
        secret:
          secretName: es-cert
          items:
            - key: elastic-certificates.p12
              path: elastic-certificates.p12

#证书与ess.yaml文件同级
[root@150m01 ~/kind]# ll
-rw-r--r-- 1 root root        3443 4月  25 09:32 elastic-certificates.p12
-rw-r--r-- 1 root root        2527 4月  25 09:33 elastic-stack-ca.p12
-rw-r--r-- 1 root root        4751 4月  25 09:35 ess.yaml
-rw-r--r-- 1 root root        4348 4月  24 18:07 ess.yaml.0

#上传ess1.yaml文件，然后应用
kubectl apply -f es1.yaml

【设置密码】

【创建多个账户】

kubectl exec -it elasticsearch-0 -n middle-ware -- bin/elasticsearch-setup-passwords interactive

Y

均为--->密码：[email protected]

Enter password for [elastic用户名首次密码]:

Reenter password for [elastic用户名确认密码]:

Enter password for [apm_system]:

Reenter password for [apm_system]:

Enter password for [kibana]:

Reenter password for [kibana]:

Enter password for [logstash_system]:

Reenter password for [logstash_system]:

Enter password for [beats_system]:

Reenter password for [beats_system]:

Enter password for [remote_monitoring_user]:

Reenter password for [remote_monitoring_user]:

【仅创建一个admin用户】

kubectl exec -it elasticsearch-0 -n middle-ware-sy -- /bin/bash

bin/elasticsearch-users useradd admin -p qqq -r superuser

【测试】

curl -u admin:qqq -X GET "http://10.10.10.150:32071/_cluster/health?pretty"

【部署报错】

有可能是因为你之前部署过es，pv和pvc可能没删除干净有残留，需要删除干净pv和pvc

=============================kibana===================================

无状态服务，展示数据，注意修改secret的账密即可，kibana的登陆页面账密也是es的账密

复制代码 
cat kibana.yml 
---
apiVersion: v1
kind: Secret
metadata:
  name: kibana-secret
  namespace: middle-ware-rp
type: Opaque
data:
  username: YWRtaW4=
  password: VllyTWs5b0Y=
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: kibana-pv
spec:
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: manual
  hostPath:
    path: /store/data/kibana
    type: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: kibana-pvc
  namespace: middle-ware-rp
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: kibana-config
  namespace: middle-ware-rp
data:
  kibana.yml: |
    server.host: "0.0.0.0"
    elasticsearch.hosts: ["http://10.10.10.133:30001"]
    xpack.security.enabled: true
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kibana
  namespace: middle-ware-rp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kibana
  template:
    metadata:
      labels:
        app: kibana
    spec:
      containers:
      - name: kibana
        image: docker.elastic.co/kibana/kibana:6.8.7
        ports:
        - containerPort: 5601
          name: http
        env:
        #- name: ELASTICSEARCH_URL
        #  value: http://elasticsearch:9200
        - name: ELASTICSEARCH_USERNAME
          valueFrom:
            secretKeyRef:
              name: kibana-secret
              key: username
        - name: ELASTICSEARCH_PASSWORD
          valueFrom:
            secretKeyRef:
              name: kibana-secret
              key: password
        volumeMounts:
        - name: kibana-config
          mountPath: /usr/share/kibana/config/kibana.yml
          subPath: kibana.yml
      volumes:
      - name: kibana-config
        configMap:
          name: kibana-config
          defaultMode: 0644
---
apiVersion: v1
kind: Service
metadata:
  name: kibana
  namespace: middle-ware-rp
spec:
  selector:
    app: kibana
  ports:
  - port: 5601
    targetPort: 5601
    name: http
    nodePort: 30003
  type: NodePort
相关推荐
不爱学英文的码字机器1 小时前
[Git] 基本操作及用户配置
大数据·git·elasticsearch
leo_hush8 小时前
python查询elasticsearch 获取指定字段的值的list
python·elasticsearch
老猿阿浪14 小时前
Git初始化本地已有项目,并推送到远端Git仓库完整操作指南
大数据·git·elasticsearch
码农爱java14 小时前
Elasticsearch 深入分析三种分页查询【Elasticsearch 深度分页】
java·大数据·spring boot·后端·elasticsearch·全文检索
java之迷16 小时前
jenkins使用Send build artifacts over SSH发布jar包目录配置
ssh·jenkins·jar
风屿.19 小时前
IDEA推送到gitlab，jenkins识别，然后自动发布到需要的主机
运维·gitlab·jenkins
XMYX-019 小时前
SkyWalking 报错：sw_profile_task 索引缺失问题分析与解决
运维·jenkins·skywalking
Elastic 中国社区官方博客19 小时前
在 JavaScript 中正确使用 Elasticsearch，第二部分
大数据·javascript·数据库·elasticsearch·搜索引擎·全文检索
predisw19 小时前
kafka 常用命令
分布式·kafka
码农爱java1 天前
Spring Boot 集成 Elasticsearch【实战】
大数据·spring boot·elasticsearch·全文检索·es
热门推荐
01【SpeedAI科研小助手】2分钟极速解决知网维普重复率、AIGC率过高，一键全文降！文件格式不变，公式都保留的！02KGG转MP3工具|非KGM文件|解密音频03从零安装 LLaMA-Factory 微调 Qwen 大模型成功及所有的坑04YOLOv8入门 | 重要性能衡量指标、训练结果评价及分析及影响mAP的因素【发论文关注的指标】05组基轨迹建模 GBTM的介绍与实现（Stata 或 R）06DeepSeek各版本说明与优缺点分析07【超详细】Windows安装Npcap08YOLOv5改进 | 添加CA注意力机制 + 增加预测层 + 更换损失函数之GIoU09Coze扣子平台完整体验和实践（附国内和国际版对比）10Anaconda +Jupyter Notebook安装（2025最新版）