小型项目elk搭建

es kibana 使用docker启动，filebeat使用本机启动

本机地址192.168.0.110

es

docker run -d --name es  -p 9200:9200 -p 9300:9300 -e "cluster.name=small-elk-cluster"  -e "node.name=es-node-1" -e "discovery.type=single-node" -e "ES_JAVA_OPTS=-Xms1g -Xmx1g" -e "xpack.security.enabled=false" -v /home/work/elk/es:/usr/share/elasticsearch/data -v /etc/localtime:/etc/localtime:ro -v /etc/timezone:/etc/timezone:ro --ulimit memlock=-1:-1 --restart always elasticsearch:7.4.2 

###这里可能es启动失败是权限问题/home/work/elk/es这个目录赋权限

chmod 777 /home/work/elk/es

kibana

docker run -d --name kibana -p 5601:5601 -e "ELASTICSEARCH_HOSTS=http://192.168.0.110:9200" -e "I18N_LOCALE=zh-CN" -e "xpack.security.enabled=false" -v /etc/localtime:/etc/localtime:ro -v /etc/timezone:/etc/timezone:ro --restart always kibana:7.4.2 

filebeat 的配置文件如下

filebeat.inputs:
- type: log
  name: "gateway"
  enabled: true
  # 订单系统日志路径（替换为你的实际路径）
  paths:
    - /home/gateway-service/logs/*.log
  # 自定义字段（可选，增强索引语义）
  fields:
    log_type: "gateway"
    env: "pro"
  fields_under_root: true
  # Java 多行日志合并（解决异常栈）
  multiline.type: pattern
  multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
  multiline.negate: true
  multiline.match: after
  encoding: utf-8
  exclude_files: ['\.zip$'] 

setup.template.name: "filebeat"
setup.template.pattern: "filebeat-*"
setup.template.settings:
  index.number_of_shards: 1
  index.number_of_replicas: 0
setup.ilm.enabled: false
# ========== 输出配置（动态生成独立索引） ==========
output.elasticsearch:
  # 指向 Docker 内的 ES 地址（替换为你的 ES 容器 IP/宿主机 IP）
  hosts: ["192.168.0.110:9200"]
  index: "filebeat-%{[log_type]}-%{+yyyy.MM.dd}"
  ilm.enabled: false
# ========== Filebeat 自身日志配置 ==========
logging.level: info
logging.to_files: true
logging.files:
  path: /usr/local/filebeat/logs
  name: filebeat
  rotateeverybytes: 10485760
  keepfiles: 7
  
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
  
processors:
  - script:
      lang: javascript
      id: my_filter
      source: >
        function process(event) {
            var str = event.Get("message");
            var time = str.split(" ").slice(0,2).join(" ");
            var standardTime = time.replace(/(\d{2}:\d{2}:\d{2}):(\d{3})$/, "$1.$2");
            event.Put("log_time", standardTime);
        }
      ignore_failure: true
  - timestamp:
      field: log_time
      target_field: "@timestamp"  # 关键：将解析的时间设为@timestamp
      timezone: Asia/Shanghai
      layouts:
        - '2006-01-02 15:04:05'
        - '2006-01-02 15:04:05.999'
      ignore_failure: true
      overwrite: true

如果时间偏移8小时，删除索引模式再删除索引,之后重新再次创建就可以了