ELK企业级日志分析平台——ES集群监控

HaoJl092023-11-24 3:04

启用xpack认证

官网：https://www.elastic.co/guide/en/elasticsearch/reference/7.6/configuring-tls.html#node-certificates

在elk1上生成证书

[root@elk1 ~]# cd /usr/share/elasticsearch/

[root@elk1 elasticsearch]# bin/elasticsearch-certutil ca

[root@elk1 elasticsearch]# bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12

[root@elk1 elasticsearch]# cp elastic-certificates.p12 /etc/elasticsearch

[root@elk1 elasticsearch]# cd /etc/elasticsearch/

[root@elk1 elasticsearch]# chown elasticsearch elastic-certificates.p12

[root@elk1 elasticsearch]# vim elasticsearch.yml

...
http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type

xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/elastic-certificates.p12

[root@elk1 elasticsearch]# systemctl  restart elasticsearch.service

拷贝证书到其它集群节点

[root@elk1 elasticsearch]# scp elastic-certificates.p12 server2:/etc/elasticsearch/

[root@elk1 elasticsearch]# scp elastic-certificates.p12 server3:/etc/elasticsearch/

elk2配置

[root@elk2 ~]# chown elasticsearch /etc/elasticsearch/elastic-certificates.p12

[root@elk2 ~]# vim /etc/elasticsearch/elasticsearch.yml

...
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/elastic-certificates.p12

[root@elk2 ~]# systemctl  restart elasticsearch.service

elk3配置

[root@elk3 ~]# chown elasticsearch /etc/elasticsearch/elastic-certificates.p12

[root@elk3 ~]# vim /etc/elasticsearch/elasticsearch.yml

...
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/elastic-certificates.p12

[root@elk3 ~]# systemctl  restart elasticsearch.service

等待所有集群节点重启成功后，设置认证账户密码

[root@elk1 elasticsearch]# cd /usr/share/elasticsearch/

[root@elk1 elasticsearch]# bin/elasticsearch-setup-passwords interactive

交互式输入密码

ES启用xpack之后，其它服务都需要配置认证

elasticsearch-head插件

head插件在访问时，需要在url中输入用户和密码

[root@k8s1 ~]# cd elasticsearch-head-master/

[root@k8s1 elasticsearch-head-master]# npm run start &

访问:192.168.92.11:9100/?auth_user=elastic&auth_password=westos

cerebro插件

cerebro访问：

[root@k8s1 ~]# docker start cerebro

访问：192.168.92.11:9000

​

kibana访问

[root@elk5 kibana]# vim kibana.yml
...
elasticsearch.username: "kibana"
elasticsearch.password: "westos"

[root@elk5 kibana]# systemctl  restart kibana.service

​

metricbeat监控

官网：https://www.elastic.co/guide/en/elasticsearch/reference/7.6/configuring-metricbeat.html

首先使用内部检测，然后进入设置模式

​

选择启动metricbeat的节点，然后根据提示步骤操作

安装软件

[root@elk1 ~]# rpm -ivh metricbeat-7.6.1-x86_64.rpm

启动elasticsearch-xpack模块

[root@elk1 ~]# cd /etc/metricbeat/modules.d

[root@elk1 modules.d]# metricbeat modules enable elasticsearch-xpack

编辑elasticsearch-xpack模块配置文件

[root@elk1 modules.d]# vim elasticsearch-xpack.yml

- module: elasticsearch
  metricsets:
    - ccr
    - cluster_stats
    - enrich
    - index
    - index_recovery
    - index_summary
    - ml_job
    - node_stats
    - shard
  period: 10s
  hosts: ["http://localhost:9200"]
  username: "remote_monitoring_user"
  password: "westos"
  xpack.enabled: true

编辑metricbeat配置文件

[root@elk1 modules.d]# cd ..

[root@elk1 metricbeat]# vim metricbeat.yml

...
output.elasticsearch:
  hosts: ["http://192.168.56.11:9200"]
  username: "elastic"
  password: "westos"

启动metricbeat服务

[root@elk1 metricbeat]# systemctl enable --now metricbeat.service

​

其它节点依次类推

​

最后退出设置模式

kibana 监控

安装软件

[root@elk5 ~]# rpm -ivh metricbeat-7.6.1-x86_64.rpm

启动kibana-xpack模块

[root@elk5 ~]# cd /etc/metricbeat/modules.d/

[root@elk5 ~]# metricbeat modules enable kibana-xpack

编辑kibana-xpack模块配置文件

[root@elk5 modules.d]# vim kibana-xpack.yml

 username: "elastic"
 password: "westos"

编辑metricbeat配置文件

[root@elk5 modules.d]# cd ..

[root@elk5 metricbeat]# vim metricbeat.yml

hosts: ["http://192.168.92.31:9200"]

username: "elastic"
password: "westos"

启动metricbeat服务

[root@elk5 ~]# systemctl enable --now metricbeat.service

禁用内部检测

[root@elk5 ~]# vim /etc/kibana/kibana.yml

xpack.monitoring.kibana.collection.enabled: false

重启kibana服务

[root@elk5 ~]# systemctl restart kibana.service

filebeat日志采集

文档：https://www.elastic.co/guide/en/beats/filebeat/7.6/filebeat-module-elasticsearch.html

安装软件

[root@elk1 ~]# rpm -ivh filebeat-7.6.1-x86_64.rpm

启动elasticsearch模块

[root@elk1 ~]# cd /etc/filebeat/modules.d

[root@elk1 modules.d]# filebeat modules enable elasticsearch

编辑elasticsearch模块配置文件

[root@elk1 modules.d]# vim elasticsearch.yml

- module: elasticsearch
  # Server log
  server:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths:
      - /var/log/elasticsearch/*.log          # Plain text logs
      - /var/log/elasticsearch/*_server.json  # JSON logs

  gc:
    enabled: true
    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths:
      - /var/log/elasticsearch/gc.log.[0-9]*
      - /var/log/elasticsearch/gc.log

  audit:
    enabled: true
    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths:
      - /var/log/elasticsearch/*_access.log  # Plain text logs
      - /var/log/elasticsearch/*_audit.json  # JSON logs

  slowlog:
    enabled: true
    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths:
      - /var/log/elasticsearch/*_index_search_slowlog.log     # Plain text logs
      - /var/log/elasticsearch/*_index_indexing_slowlog.log   # Plain text logs
      - /var/log/elasticsearch/*_index_search_slowlog.json    # JSON logs
      - /var/log/elasticsearch/*_index_indexing_slowlog.json  # JSON logs

  deprecation:
    enabled: true
    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths:
      - /var/log/elasticsearch/*_deprecation.log   # Plain text logs
      - /var/log/elasticsearch/*_deprecation.json  # JSON logs

编辑filebeat配置文件

[root@elk1 modules.d]# cd ..

[root@elk1 filebeat]# vim filebeat.yml

...
output.elasticsearch:
  hosts: ["http://192.168.92.31:9200"]
  username: "elastic"
  password: "westos"

启动filebeat服务

[root@elk1 filebeat]# systemctl  enable --now filebeat.service

​

其它节点依次类推

​

相关推荐
鸡c29 分钟前
es的封装
大数据·elasticsearch·搜索引擎
Elastic 中国社区官方博客4 小时前
Elasticsearch：检索增强生成背后的重要思想
大数据·人工智能·elasticsearch·搜索引擎·全文检索
中间件XL4 小时前
搜索引擎onesearch3实现解释和升级到Elasticsearch v8系列(四)-搜索
elasticsearch·搜索引擎·全文检索·elasticsearch8
Fan6 小时前
Elasticsearch 下载安装及使用总结
大数据·elasticsearch·jenkins
TO_ZRG8 小时前
使用jenkins打包unity工程
运维·unity·jenkins
必叫你大败而归8 小时前
ELK日志分析系统
elk
liujiangxu8 小时前
jenkins声明式流水线语法详解
运维·自动化·jenkins
Connie14518 小时前
filebeat采集挂载出来的/home/Logs下的日志过程
elasticsearch
確定饿的猫9 小时前
ELK环境部署
elk
YCyjs9 小时前
ELK 企业级日志分析系统
elk
热门推荐
01CCF-CSP认证考试 202406-3 文本分词 100分题解02RAG 实践- Ollama+RagFlow 部署本地知识库032024年高教社杯数学建模国赛C题超详细解题思路分析04实战 | 基于YOLOv9和OpenCV实现车辆跟踪计数（步骤 + 源码）05组基轨迹建模 GBTM的介绍与实现（Stata 或 R）06【2024数模国赛赛题思路公开】国赛B题思路丨附可运行代码丨无偿自提07软件工程从理论到实践客观题汇总(头歌第一章至第八章)08苍穹外卖面试总结09【2024高教社杯全国大学生数学建模竞赛】B题 生产过程中的决策问题——解题思路 代码 论文10yolov8实战第五天——yolov8+ffmpg实时视频流检测并进行实时推流——（推流，保姆教学）