Elasticsearch安装使用

JDK安装

1.上传jdk安装包到/opt临时目录

2.解压安装

复制代码

cd /opt
tar xvf jdk-8u192-linux-x64.tar.gz
mkdir /usr/java
mv jdk1.8.0_192/ /usr/java/jdk1.8

3.在profile文件末添加环境变量

复制代码

#root用户
vi /etc/profile
export JAVA_HOME=/usr/java/jdk1.8
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=.:$JAVA_HOME/bin:$PATH

4.刷新环境变量

复制代码

source /etc/profile

5.验证JDK安装成功与否

复制代码

java -version

ELK安装

1.安装ElasticSearch

修改最大虚拟内存映射

复制代码

vim /etc/sysctl.conf
添加如下配置
vm.max_map_count = 655360

使配置生效

/sbin/sysctl -p

创建目录

复制代码

cd /usr/local/fs
mkdir elasticsearch
cd elasticsearch

上传包并解压

复制代码

tar -xvf elasticsearch-7.17.27-linux-aarch64.tar.gz
cd elasticsearch-7.17.27

修改配置

vim config/elasticsearch.yml

文件内增加如下内容

复制代码

network.host: 0.0.0.0
cluster.initial_master_nodes: ["node-1"]
node.name: node-1

添加es用户

复制代码

useradd es
passwd es
SZtest898
SZtest898
chown -R es:es /usr/local/fs/elasticsearch/elasticsearch-7.17.27

后台启动es

复制代码

su es
cd bin
nohup ./elasticsearch &

2.Kibana安装

上传相同版本的包并解压

复制代码

mkdir kibana
tar -xvf kibana-7.17.27-linux-aarch64.tar.gz
cd kibana-7.17.27-linux-aarch64

修改配置文件

复制代码

vim config/kibana.yml
增加如下内容
#允许远程访问
server.host: "0.0.0.0"
#远程访问路径
server.publicBaseUrl: "http://x.x.x.x:5601/"
#elasticsearch主机地址
elasticsearch.host: ["http://localhost:9200"]
#elasticsearch程序启动用户名和密码
elasticsearch.username: "es"
elasticsearch.password: "SZtest898"
i18n.locale: "zh-CN"

启动kibana

复制代码

chown -R es:es /usr/local/fs/kibana/kibana-7.17.27-linux-aarch64
su es
nohup bin/kibana &

3.安装Logstash

上传相同版本的包并解压

复制代码

mkdir logstash
tar -xvf logstash-7.17.27-linux-aarch64.tar.gz
cd logstash-7.17.27

修改配置文件logstash.conf（参考配置）

复制代码

input {
  beats {
    port => 5044
  }
}

filter {
  multiline {
    pattern => "^\[\d{8} \d{2}:\d{2}:\d{2}\.\d{3}\]"
    negate => true
    what => "previous"
  }
}

output {
  if "fs-cop-admin" in [tags] {
     elasticsearch {
          hosts => ["http://10.201.69.87:9200"]
          index => "%{[fields][service_name]}-%{+YYYY.MM.dd}"
  	 }
   }
  if "gx-kcop-basic-app" in [tags] {
     elasticsearch {
          hosts => ["http://10.201.69.87:9200"]
          index => "%{[fields][service_name]}-%{+YYYY.MM.dd}"
  	 }
   }
  if "gx-kcop-biz-app" in [tags] {
     elasticsearch {
          hosts => ["http://10.201.69.87:9200"]
          index => "%{[fields][service_name]}-%{+YYYY.MM.dd}"
  	 }
   }
  if "gx-kcop-kgds-app" in [tags] {
     elasticsearch {
          hosts => ["http://10.201.69.87:9200"]
          index => "%{[fields][service_name]}-%{+YYYY.MM.dd}"
  	 }
   }
  if "gx-kcop-query-app" in [tags] {
     elasticsearch {
          hosts => ["http://10.201.69.87:9200"]
          index => "%{[fields][service_name]}-%{+YYYY.MM.dd}"
  	 }
   }
  if "gx-kcop-scheduler-app" in [tags] {
     elasticsearch {
          hosts => ["http://10.201.69.87:9200"]
          index => "%{[fields][service_name]}-%{+YYYY.MM.dd}"
  	 }
   }
}

安装multiline插件

复制代码

bin/logstash-plugin install logstash-filter-multiline

启动logstash

复制代码

chown -R es:es /usr/local/fs/logstash/logstash-7.17.27
su es
nohup bin/logstash -f config/logstash.conf --path.data data &
nohup ./logstash -f config/logstash.conf --path.data data &

4.要收集日志的Filebeat

此步骤也可以省略，logstash也可直接收集日志文件，但是不如filebeat轻量，filebeat可以占用更少的资源部署在微服务上，然后将本地日志文件传给logstash处理

上传相同版本的包并解压

复制代码

mkdir filebeat
tar -xvf filebeat-*.tar.gz
cd filebeat-7.17.27-linux-arm64

修改配置文件

复制代码

vim filebeat.yml
修改如下内容
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/*.log
  tags: ["fs-cop-admin"]
output.logstash:
  hosts: ["localhost:5044"]

参考配置

复制代码

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /usr/local/fs/kcop-basic-app/logs/*.log
  tags: ["gx-kcop-basic-app"]
  fields:
    service_name: gx-kcop-basic-app
- type: log
  enabled: true
  paths:
    - /usr/local/fs/kcop-biz-app/logs/*.log
  tags: ["gx-kcop-biz-app"]
  fields:
    service_name: gx-kcop-biz-app
- type: log
  enabled: true
  paths:
    - /usr/local/fs/kcop-kgds-app/logs/*.log
  tags: ["gx-kcop-kgds-app"]
  fields:
    service_name: gx-kcop-kgds-app
- type: log
  enabled: true
  paths:
    - /usr/local/fs/kcop-query-app/logs/*.log
  tags: ["gx-kcop-query-app"]
  fields:
    service_name: gx-kcop-query-app
- type: log
  enabled: true
  paths:
    - /usr/local/fs/kcop-scheduler-app/logs/*.log
  tags: ["gx-kcop-scheduler-app"]
  fields:
    service_name: gx-kcop-scheduler-app
output.logstash:
  hosts: ["localhost:5044"]

启动filebeat

复制代码

chown -R es:es /usr/local/fs/filebeat/filebeat-7.17.27-linux-arm64/
su es
nohup ./filebeat -e -c filebeat.yml > filebeat.out &

5.启动后常见问题处理

访问elasticsearch不上

复制代码

#1.可以通过ping或者telnet命令查看一下本地机器与服务器地址是否想通
ping 10.201.65.84
telnet 10.201.65.84 9200

#2.可以在服务器上利用curl -v命令看下服务是否正常，如果是正常的话，说明服务启动没有问题
curl -v http://10.201.65.84:9200

#3.如果都是通的，查看elasticsearch日志服务也已经启动，就要考虑部署elasticsearch的服务器是否关闭了防火墙，如果输出状态是not running->关闭状态|running->启用状态。
sudo firewall-cmd --state
#3.1如果是启用状态，尝试关闭防火墙
sudo systemctl stop firewalld

启动logstash报错

复制代码

#报错信息如下
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[FATAL] 2025-02-28 15:25:36.995 [main] Logstash - Logstash was unable to start due to an unexpected Gemfile change.
If you are a user, this is a bug.
If you are a logstash developer, please try restarting logstash with the `--enable-local-plugin-development` flag set.

#这一般就是下载插件那一步没成功导致的，下载插件即可，这一步下载可能会比较久，需要确定下载后，输出了sucess信息才行
bin/logstash-plugin install logstash-filter-multiline

启动了filebeat，配置文件都正确，界面操作找不到索引

复制代码

#出现这种问题，一般是filebeat没有搭建在相对应的服务器上。