ELK企业级日志分析平台——elasticsearch

集群部署

文档：https://www.elastic.co/guide/en/elasticsearch/reference/7.6/index.html

|------|---------------|---------------|
| 主机 | ip | 角色 |
| k8s1 | 192.168.92.11 | cerebro |
| elk1 | 192.168.92.31 | elasticsearch |
| elk2 | 192.168.92.32 | elasticsearch |
| elk3 | 192.168.92.33 | elasticsearch |
| elk4 | 192.168.92.34 | logstash |
| elk5 | 192.168.92.35 | kibana |

配置解析

复制代码

[root@elk1 ~]# cat /etc/hosts

软件安装

复制代码

[root@elk1 ~]# rpm -ivh elasticsearch-7.6.1-x86_64.rpm

修改配置

复制代码

[root@elk1 ~]# cd /etc/elasticsearch/

[root@elk1 elasticsearch]# vim elasticsearch.yml

cluster.name: my-es

network.host: 0.0.0.0

http.port: 9200

discovery.seed_hosts: ["elk1", "elk2", "elk3"]

cluster.initial_master_nodes: ["elk1", "elk2", "elk3"]

系统设置

复制代码

[root@elk1 ~]# vim /etc/security/limits.conf

elasticsearch soft memlock unlimited
elasticsearch hard memlock unlimited
elasticsearch  -  nofile  65535
elasticsearch  -  nproc  4096

[root@elk1 ~]# vim /usr/lib/systemd/system/elasticsearch.service

[service]
...
LimitMEMLOCK=infinity

[root@elk1 ~]# systemctl  daemon-reload

[root@elk1 ~]# swapoff -a

[root@elk1 ~]# vim /etc/fstab

复制代码

[root@elk1 ~]# systemctl daemon-reload

[root@elk1 ~]# systemctl  enable --now  elasticsearch

elk1配置好后，直接把配置复制到elk2和elk3

配置ssh免密

复制代码

[root@elk1 ~]# ssh-keygen

[root@elk1 ~]# ssh-copy-id server2

[root@elk1 ~]# ssh-copy-id server3

复制软件

复制代码

[root@elk1 ~]# scp elasticsearch-7.6.1-x86_64.rpm server2:

[root@elk1 ~]# scp elasticsearch-7.6.1-x86_64.rpm server3:

elk2和elk3软件安装

复制代码

[root@elk2 ~]# rpm -ivh elasticsearch-7.6.1-x86_64.rpm

[root@elk3 ~]# rpm -ivh elasticsearch-7.6.1-x86_64.rpm

从elk1复制配置

复制代码

[root@elk1 ~]# cd /etc/elasticsearch/

[root@elk1 elasticsearch]# scp elasticsearch.yml elk2:/etc/elasticsearch/

[root@elk1 elasticsearch]# scp elasticsearch.yml elk3:/etc/elasticsearch/

[root@elk1 elasticsearch]# scp /etc/security/limits.conf elk2:/etc/security/

[root@elk1 elasticsearch]# scp /etc/security/limits.conf elk3:/etc/security/

[root@elk1 elasticsearch]# scp /usr/lib/systemd/system/elasticsearch.service elk2:/usr/lib/systemd/system/

[root@elk1 elasticsearch]# scp /usr/lib/systemd/system/elasticsearch.service elk3:/usr/lib/systemd/system/

server2上启动服务

复制代码

[root@elk2 ~]# swapoff -a

[root@elk2 ~]# vim /etc/fstab

复制代码

[root@elk2 ~]# systemctl daemon-reload

[root@elk2 ~]# systemctl  enable --now  elasticsearch

server3上启动服务

复制代码

[root@elk3 ~]# swapoff -a

[root@elk3 ~]# vim /etc/fstab

复制代码

[root@elk3 ~]# systemctl daemon-reload

[root@elk3 ~]# systemctl  enable --now  elasticsearch

cerebro部署

cerebro官方：https://github.com/lmenezes/cerebro/

使用docker启动服务

复制代码

[root@k8s1 ~]#  docker pull lmenezes/cerebro

[root@k8s1 ~]#  docker run -d --name cerebro -p 9000:9000 lmenezes/cerebro

访问网页：192.168.92.11:9000

节点地址可以填写任意ES集群节点ip

elasticsearch集群角色分类

复制代码

[root@elk1 ~]# vim /etc/elasticsearch/elasticsearch.yml

node.master: true
node.data: false
node.ingest: true
node.ml: false

[root@elk1 ~]# systemctl  restart elasticsearch.service

复制代码

[root@elk2 ~]# vim /etc/elasticsearch/elasticsearch.yml

node.master: true
node.data: true
node.ingest: false
node.ml: false

[root@elk2 ~]# systemctl  restart elasticsearch.service

复制代码

[root@elk3 ~]# vim /etc/elasticsearch/elasticsearch.yml

node.master: true
node.data: true
node.ingest: false
node.ml: false

[root@elk3 ~]# systemctl  restart elasticsearch.service