夜莺浏览日志、filebeat采集日志（四）

文章目录

• 一、elasticsearch
• 二、filebeat
• 三、日志分析

---

一、elasticsearch

• docker启动

docker run -d -p 9200:9200 -p 9300:9300 --restart=always -e ES_JAVA_OPTS="-Xms512m -Xmx512m" \
-e discovery.type=single-node -e xpack.security.enabled=true -e ELASTIC_PASSWORD=123456 \
-v /home/monitor/elasticsearch/data:/usr/share/elasticsearch/data --name monitor-es elasticsearch:8.12.2

• 用户名 elastic，密码 123456

二、filebeat

• 下载地址
  filebeat-8.12.2-linux-x86_64.tar.gz

• Linux目录

  /home/monitor/filebeat
  

  filebeat文件来源于filebeat-8.12.2-linux-x86_64.tar.gz

• filebeat.service

[Unit]
Description=Filebeat
After=network.target
 
[Service]
Type=simple
ExecStart=/home/monitor/filebeat/filebeat -e -c /home/monitor/filebeat/filebeat.yml

[Install]
WantedBy=multi-user.target

• filebeat.yml
  配置需要监控的日志，例如nginx、redis，配置写入的elasticsearch信息

filebeat.inputs:
- type: filestream
  paths:
    - /home/nginx/logs/access.log
  prospector.scanner.exclude_files: ['.gz$']
  tags: ["nginx"]

- type: filestream
  paths:
    - /home/logs/example/all.log
  prospector.scanner.exclude_files: ['.gz$']
  tags: ["example"]

output.elasticsearch:
  hosts: ["192.168.6.12:9200"]
  preset: balanced
  protocol: "http"
  username: "elastic"
  password: "123456"
  indices:
    - index: "filebeat-6.13-%{+yyyy.MM}"

setup.template.settings:
  index.number_of_shards: 1
  index.codec: best_compression

processors:
  - drop_fields:
      fields: ["log","host","input","agent","ecs"]
      ignore_missing: false

• filebeat服务安装

chmod 755 /home/monitor/filebeat/filebeat.yml
chmod 777 /home/monitor/filebeat/filebeat
cp /home/monitor/filebeat/filebeat.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl start filebeat && sudo systemctl enable filebeat

• 检查服务状态

sudo systemctl status filebeat

三、日志分析

• 配置elasticsearch
  系统配置 > 数据源 > elasticsearch

• 日志分析
  日志分析 > 即时查询
  展示字段：tags、message
  过滤条件例子：tags:example AND message:INFO