Docker Compose搭建ELK
Elasticsearch默认使用mmapfs目录来存储索引。操作系统默认的mmap计数太低可能导致内存不足,我们可以使用下面这条命令来增加内存
shell
sysctl -w vm.max_map_count=262144
创建Elasticsearch数据挂载路径
shell
mkdir -p /echola/elasticsearch/data
对该路径授予777权限
shell
chmod 777 /echola/elasticsearch/data
创建Elasticsearch插件挂载路径
shell
mkdir -p /echola/elasticsearch/plugins
创建Logstash配置文件存储路径
shell
mkdir -p /echola/logstash
在该路径下创建logstash-echola.conf配置文件(没有vim命令,可以使用yum install vim
命令安装)
shell
vi /echola/logstash/logstash-echola.conf
内容如下
shell
input {
tcp {
mode => "server"
host => "0.0.0.0"
port => 4560
codec => json_lines
}
}
output {
elasticsearch {
hosts => "es:9200"
index => "echola-logstash-%{+YYYY.MM.dd}"
}
}
创建ELK Docker Compose文件存储路径:
shell
mkdir -p /echola/elk
在ellk目录下创建docker-compose.yml
文件
shell
vi /echola/elk/docker-compose.yml
内容如下:
yml
version: '3'
services:
elasticsearch:
image: elasticsearch:6.4.1
container_name: elasticsearch
environment:
- "cluster.name=elasticsearch" #集群名称为elasticsearch
- "discovery.type=single-node" #单节点启动
- "ES_JAVA_OPTS=-Xms512m -Xmx512m" #jvm内存分配为512MB
volumes:
- /echola/elasticsearch/plugins:/usr/share/elasticsearch/plugins
- /echola/elasticsearch/data:/usr/share/elasticsearch/data
ports:
- 9200:9200
kibana:
image: kibana:6.4.1
container_name: kibana
links:
- elasticsearch:es #配置elasticsearch域名为es
depends_on:
- elasticsearch
environment:
- "elasticsearch.hosts=http://es:9200" #因为上面配置了域名,所以这里可以简写为http://es:9200
ports:
- 5601:5601
logstash:
image: logstash:6.4.1
container_name: logstash
volumes:
- /echola/logstash/logstash-echola.conf:/usr/share/logstash/pipeline/logstash.conf
depends_on:
- elasticsearch
links:
- elasticsearch:es
ports:
- 4560:4560
切换到 /echola/elk
,启动docker-compose
shell
docker-compose up -d
可能会出现权限不够:-bash: /usr/local/bin/docker-compose: 权限不够
shell
chmod +x /usr/local/bin/docker-compose
ElasticSerach启动报错:
java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes
以为是 /usr/share/elasticsearch/data/nodes目录权限不够,其实是宿主机上的/echola/elasticsearch/data目录权限不足造成
shell
chmod 777 /echola/elasticsearch/data
Failed to clear cache for realms [[]]
清除区域缓存失败,刷新一下网页即可
补充:
访问Kibana,显示:Cannot connect to the Elasticsearch cluster currently configured for Kibana
重启Docker
shell
systemctl restart docker
Logstash中安装json_lines插件
使用如下命令进入到Logstash容器中
shell
docker exec -it logstash /bin/bash
切换到/bin目录,安装json_lines插件,然后退出
shell
cd /bin/
logstash-plugin install logstash-codec-json_lines
可能会出现以下错误:内存不够,先关掉ES和Kibana
然后访问IP地址:http://192.168.2.203:5601/
可能存在防火墙,导致网站访问不了
查看防火墙状态
shell
systemctl status firewalld
停止防火墙
shell
systemctl stop firewalld
禁止防火墙开机启动
shell
systemctl disable firewalld
或者指定开放端口
shell'
systemctl start firewalld
开放端口
shell
firewall-cmd --zone=public --add-port=5601/tcp --permanent
重新加载
shell
firewall-cmd --reload
Kibana图示: