1.什么是ELK
ELK是Elasticsearch、Logstash、 Kibana三大开源框架首字母大写简称。市面上也被成为Elastic Stack
1.Elasticsearch 负责日志的检索和存储
2.Logstash 负责日志的收集和分析,处理
3.Kibana 负责日志的可视化
官方文档说明启动 Elasticsearch |Elasticsearch 指南 [8.11] |弹性的
ELK架构:
1.搭建ELK1.先搭建 Elasticsearsh ,因为搭建了 Elasticsearsh之后,无论是搭建 Kibana,还是 logstash都可以尽兴数的联调
2.搭建Kibana ,先把 web界面展示出来
3.搭建 logstash
1.安装 ElasticsearshES最新下载地址 :Past Releases of Elastic Stack Software | Elastic
历史版本下载 :Past Releases of Elastic Stack Software | Elastic
官网的下载会非常慢,我们也可以选择华为云镜像下载 :Index of elasticsearch-local
1.下载好之后会有一个包,我们把他传到服务器上
2.此时就已经传到服务器上了,并解压
bash
[root@xpxplinux ~]# ls
anaconda-ks.cfg elasticsearch-8.11.3-linux-x86_64.tar.gz initial-setup-ks.cfg snap 公共 模板 视频 图片 文档 下载 音乐 桌面
[root@xpxplinux ~]# tar xf elasticsearch-8.11.3-linux-x86_64.tar.gz
[root@xpxplinux ~]# ls
anaconda-ks.cfg elasticsearch-8.11.3 elasticsearch-8.11.3-linux-x86_64.tar.gz initial-setup-ks.cfg snap 公共 模板 视频 图片 文档 下载 音乐 桌面
[root@xpxplinux ~]# cd elasticsearch-8.11.3/
[root@xpxplinux elasticsearch-8.11.3]# ls
bin config jdk lib LICENSE.txt logs modules NOTICE.txt plugins README.asciidoc
#目录介绍
bin 启动文件
config 配置文件目录
log4j2 日志配置文件
jvm.options java虚拟机相关的配置(默认启动占1g内存,内容不够需要自己调整)
elasticsearch.yml elasticsearch的配置文件! 默认9200端口!跨域!
lib 相关jar包
modules 功能模块目录
plugins 插件目录 ik分词器
3.更改配置文件
bash
[root@xpxplinux config]# pwd
/root/elasticsearch-8.11.3/config
[root@xpxplinux config]# vim elasticsearch.yml
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
network.host: 0.0.0.0
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
#吧 Network 的network.host 改成 0.0.0.0 因为是yaml语法,所以必须是顶行写 负责会报错
4.启动
bash
[root@xpxplinux bin]# pwd
/root/elasticsearch-8.11.3/bin
[root@xpxplinux bin]# ./elasticsearch
十二月 16, 2023 11:58:18 上午 sun.util.locale.provider.LocaleProviderAdapter <clinit>
WARNING: COMPAT locale provider will be removed in a future release
[2023-12-16T11:58:20,648][ERROR][o.e.b.Elasticsearch ] [xpxplinux] fatal exception while booting Elasticsearchjava.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.server@8.11.3/org.elasticsearch.bootstrap.Elasticsearch.initializeNatives(Elasticsearch.java:282)
at org.elasticsearch.server@8.11.3/org.elasticsearch.bootstrap.Elasticsearch.initPhase2(Elasticsearch.java:167)
at org.elasticsearch.server@8.11.3/org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:72)
See logs for more details.
ERROR: Elasticsearch did not exit normally - check the logs at /root/elasticsearch-8.11.3/logs/elasticsearch.log
ERROR: Elasticsearch exited unexpectedly, with exit code 1
当报这个错的时候,是因为 elasticsearch 是不允许root用户执行
5.创建用户,并执行
bash
[root@xpxplinux ~]# useradd es
[root@xpxplinux ~]# passwd es
[root@xpxplinux ~]# cp -r elasticsearch-8.11.3 /home/es/
[root@xpxplinux ~]# chmod -R 777 /home/es/elasticsearch-8.11.3/
[root@xpxplinux ~]# su - es
上一次登录:六 12月 16 21:42:36 CST 2023pts/0 上
[es@xpxplinux ~]$ cd elasticsearch-8.11.3/bin/
[es@xpxplinux bin]$ ls
elasticsearch elasticsearch-create-enrollment-token elasticsearch-geoip elasticsearch-reconfigure-node elasticsearch-setup-passwords elasticsearch-syskeygen
elasticsearch-certgen elasticsearch-croneval elasticsearch-keystore elasticsearch-reset-password elasticsearch-shard elasticsearch-users
elasticsearch-certutil elasticsearch-env elasticsearch-node elasticsearch-saml-metadata elasticsearch-sql-cli
elasticsearch-cli elasticsearch-env-from-file elasticsearch-plugin elasticsearch-service-tokens elasticsearch-sql-cli-8.11.3.jar
[es@xpxplinux bin]$ ./elasticsearch
[2023-12-16T21:52:12,873][ERROR][o.e.b.Elasticsearch ] [xpxplinux] node validation exception
[1] bootstrap checks failed. You must address the points described in the following [1] lines before starting Elasticsearch. For more information see [https://www.elastic.co/guide/en/elasticsearch/reference/8.11/bootstrap-checks.html]
bootstrap check failure [1] of [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]; for more information see [https://www.elastic.co/guide/en/elasticsearch/reference/8.11/_maximum_map_count_check.html]
ERROR: Elasticsearch did not exit normally - check the logs at /home/es/elasticsearch-8.11.3/logs/elasticsearch.log
[2023-12-16T21:52:12,918][WARN ][o.e.n.Node ] [xpxplinux] unexpected exception while waiting for http server to closejava.util.concurrent.ExecutionException: java.lang.IllegalStateException: Can't move to stopped state when not started
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at org.elasticsearch.server@8.11.3/org.elasticsearch.node.Node.prepareForClose(Node.java:1776)
at org.elasticsearch.server@8.11.3/org.elasticsearch.bootstrap.Elasticsearch.shutdown(Elasticsearch.java:468)
at java.base/java.lang.Thread.run(Thread.java:1583)
See logs for more details.
发现抱错
6.修改系统最大文件句柄数(修改后需要重启系统才能生效)
bash
[root@xpxplinux ~]# vim /etc/sysctl.conf
[root@xpxplinux ~]# grep -Ev '^$|^#' /etc/sysctl.conf
vm.max_map_count=262144
[root@xpxplinux ~]# sysctl -p
vm.max_map_count = 262144
[root@xpxplinux ~]# su - es
上一次登录:六 12月 16 21:56:03 CST 2023:0 上
[es@xpxplinux ~]$ cd elasticsearch-8.11.3/bin/
[es@xpxplinux bin]$ ls
elasticsearch elasticsearch-create-enrollment-token elasticsearch-geoip elasticsearch-reconfigure-node elasticsearch-setup-passwords elasticsearch-syskeygen
elasticsearch-certgen elasticsearch-croneval elasticsearch-keystore elasticsearch-reset-password elasticsearch-shard elasticsearch-users
elasticsearch-certutil elasticsearch-env elasticsearch-node elasticsearch-saml-metadata elasticsearch-sql-cli
elasticsearch-cli elasticsearch-env-from-file elasticsearch-plugin elasticsearch-service-tokens elasticsearch-sql-cli-8.11.3.jar
[es@xpxplinux bin]$ ./elasticsearch
[2023-12-16T22:07:14,233][WARN ][o.e.h.n.Netty4HttpServerTransport] [xpxplinux] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/192.168.134.139:9200, remoteAddress=/192.168.134.139:42332}
[2023-12-16T22:07:41,327][WARN ][o.e.h.n.Netty4HttpServerTransport] [xpxplinux] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/192.168.134.139:9200, remoteAddress=/192.168.134.1:53986}
[2023-12-16T22:07:41,328][WARN ][o.e.h.n.Netty4HttpServerTransport] [xpxplinux] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/192.168.134.139:9200, remoteAddress=/192.168.134.1:53987}
#此时抱这个错误是因 导致 localhost:9200 无法访问,是因为 elasticsearch 开启了安全认证模式,需要关闭
- 关闭 elasticsearch 安全认证模式,打开vim elasticsearch.yml,吧这两个 ture改成 false,保存退出
8.重启 elasticsearch
bash
[es@xpxplinux config]$ vim elasticsearch.yml
[es@xpxplinux config]$ cd ..
[es@xpxplinux elasticsearch-8.11.3]$ cd bin/
[es@xpxplinux bin]$ ls
elasticsearch elasticsearch-create-enrollment-token elasticsearch-geoip elasticsearch-reconfigure-node elasticsearch-setup-passwords elasticsearch-syskeygen
elasticsearch-certgen elasticsearch-croneval elasticsearch-keystore elasticsearch-reset-password elasticsearch-shard elasticsearch-users
elasticsearch-certutil elasticsearch-env elasticsearch-node elasticsearch-saml-metadata elasticsearch-sql-cli
elasticsearch-cli elasticsearch-env-from-file elasticsearch-plugin elasticsearch-service-tokens elasticsearch-sql-cli-8.11.3.jar
[es@xpxplinux bin]$ ./elasticsearch
十二月 16, 2023 10:18:28 下午 sun.util.locale.provider.LocaleProviderAdapter <clinit>
WARNING: COMPAT locale provider will be removed in a future release
[2023-12-16T22:18:30,173][INFO ][o.a.l.i.v.PanamaVectorizationProvider] [xpxplinux] Java vector incubator API enabled; uses preferredBitSize=256
[2023-12-16T22:18:31,121][INFO ][o.e.n.Node ] [xpxplinux] version[8.11.3], pid[4973], build[tar/64cf052f3b56b1fd4449f5454cb88aca7e739d9a/2023-12-08T11:33:53.634979452Z], OS[Linux/3.10.0-693.el7.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/21.0.1/21.0.1+12-29]
9.再次访问