如何使用 Graylog 连接 Easysearch

Graylog 是一款功能强大的开源日志管理平台，专为日志的聚合、存储、分析和告警而设计。它能够从各种数据源实时采集日志，并通过直观的 Web 界面提供高效的全文搜索、仪表板构建和基于条件的告警功能，帮助开发者和运维团队快速定位问题、监控系统状态并保障业务稳定运行，是企业级日志处理的解决方案之一。

安装

为了演示，本篇采用 docker compose 的方式直接拉起 Graylog 和 Easysearch 相关的服务。

复制代码

services:
  mongodb:
    image: "mongo:6.0.18"
    ports:
      - "27017:27017"
    restart: "on-failure"
    networks:
      - graylog
    volumes:
      - "mongodb_data:/data/db"

  easysearch:
    image: "infinilabs/easysearch:1.13.1-2180"
    environment:
      - "cluster.name=es1"
      - "ES_JAVA_OPTS=-Xms1g -Xmx1g"
      - "bootstrap.memory_lock=true"
      - "EASYSEARCH_INITIAL_ADMIN_PASSWORD=changeme"
      - "elasticsearch.api_compatibility=true"
      - "security.ssl.http.enabled=false"
    ulimits:
      memlock:
        hard: -1
        soft: -1
      nofile:
        soft: 65536
        hard: 65536
    ports:
      - "9222:9200"
      #- "9300:9300"
    restart: "on-failure"
    networks:
      - graylog
    volumes:
      - "easysearch_data:/usr/share/easysearch/data"


  # Graylog: https://hub.docker.com/r/graylog/graylog/
  graylog:
    hostname: "server"
    image: "graylog/graylog-enterprise:6.0"
    # To install Graylog Open: "graylog/graylog:6.0"
    depends_on:
      mongodb:
        condition: "service_started"
    entrypoint: "/usr/bin/tini -- wait-for-it easysearch:9200 -- /docker-entrypoint.sh"
    environment:
      GRAYLOG_NODE_ID_FILE: "/usr/share/graylog/data/config/node-id"
      GRAYLOG_HTTP_BIND_ADDRESS: "0.0.0.0:9090"
      GRAYLOG_ELASTICSEARCH_HOSTS: "http://admin:changeme@easysearch:9200"
      GRAYLOG_MONGODB_URI: "mongodb://mongodb:27017/graylog"
      # To make reporting (headless_shell) work inside a Docker container
      GRAYLOG_REPORT_DISABLE_SANDBOX: "true"
      # CHANGE ME (must be at least 16 characters)!
      GRAYLOG_PASSWORD_SECRET: "somepasswordpepper"
      # Password: "admin"
      GRAYLOG_ROOT_PASSWORD_SHA2: "8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918"
      GRAYLOG_HTTP_EXTERNAL_URI: "http://127.0.0.1:9090/"
    ports:
      # Test config
      - "5555:5555/tcp"
      # Graylog web interface and REST API
      - "9090:9090/tcp"
      # Beats
      - "5044:5044/tcp"
      # Syslog TCP
      - "5140:5140/tcp"
      # Syslog UDP
      - "5140:5140/udp"
      # GELF TCP
      - "12201:12201/tcp"
      # GELF UDP
      - "12201:12201/udp"
      # Forwarder data
      - "13301:13301/tcp"
      # Forwarder config
      - "13302:13302/tcp"
    restart: "on-failure"
    networks:
      - graylog
    volumes:
      - "graylog_data:/usr/share/graylog/data/data"
      - "graylog_config:/usr/share/graylog/data/config"
      - "graylog_journal:/usr/share/graylog/data/journal"

networks:
  graylog:
    driver: "bridge"

volumes:
  mongodb_data:
  easysearch_data:
  graylog_data:
  graylog_config:
  graylog_journal:

测试

服务启动后访问 Graylog Web 控制台。

建立用于数据摄入的 INPUT 绑定 5555 端口。

建立 INDEXSET 用于存储数据。

后端 Easysearch 自动创建了对应的索引 test-index_0。

最后在 Graylog 建立 STREAMS 用于从 INPUT 接收数据后存入对应的 INDEXSET。

为 test STREAM 添加规则，只写入以 "message" 结尾的数据。

最后点击启动该 STREAM。

从终端向本机 5555 端口发送测试数据。

复制代码

echo 'First log message' | nc localhost 5555
echo 'Hello' | nc localhost 5555
echo 'Second log message' | nc localhost 5555

查看 test stream 中的数据，只有以 message 结尾的。