使用logstash和elasticsearch实现日志链路（一）

1.安装环境

elasticsearch：

logstash：

2.需求

2.1.需求描述

对于中台多个微服务的一个交易链路，能够全部串联查询出来。在中台前端显示。

2.2.日志样例

bash 复制代码

[linkservera] [695ca6b346e9e8d63bf96eda44f09c6e,3bf96eda44f09c6e] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:47.309] > [INFO] [http-nio-13000-exec-1] - ========================================== 开始请求 ==========================================
[linkservera] [695ca6b346e9e8d63bf96eda44f09c6e,3bf96eda44f09c6e] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:47.318] > [INFO] [http-nio-13000-exec-1] - 请求URL: http://localhost:13000/api/common/used
[linkservera] [695ca6b346e9e8d63bf96eda44f09c6e,3bf96eda44f09c6e] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:47.318] > [INFO] [http-nio-13000-exec-1] - 调用方法: POST
[linkservera] [695ca6b346e9e8d63bf96eda44f09c6e,3bf96eda44f09c6e] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:47.318] > [INFO] [http-nio-13000-exec-1] - 请求IP: 127.0.0.1
[linkservera] [695ca6b346e9e8d63bf96eda44f09c6e,3bf96eda44f09c6e] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:47.319] > [INFO] [http-nio-13000-exec-1] - 请求类方法: com.gg.midend.controller.api.CommonApiController.getProcessMemory
[linkservera] [695ca6b346e9e8d63bf96eda44f09c6e,3bf96eda44f09c6e] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:47.320] > [INFO] [http-nio-13000-exec-1] - 请求参数: {"tradeTime":"2026-03-22 08:18:00","name":"老王","patientId":"123456"}
[linkservera] [695ca6b346e9e8d63bf96eda44f09c6e,3bf96eda44f09c6e] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:47.955] > [INFO] [http-nio-13000-exec-1] - 耗时: 634毫秒
[linkservera] [695ca6b346e9e8d63bf96eda44f09c6e,3bf96eda44f09c6e] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:47.956] > [INFO] [http-nio-13000-exec-1] - 返回结果: {"tradeTime":"2026-03-22 08:18:00","name":"老王","patientId":"123456"}
[linkservera] [695ca6b346e9e8d63bf96eda44f09c6e,3bf96eda44f09c6e] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:47.956] > [INFO] [http-nio-13000-exec-1] - ========================================== 请求结束 ==========================================

[linkservera] [695ca6b528140ca7802052bea3187450,802052bea3187450] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:49.491] > [INFO] [http-nio-13000-exec-2] - ========================================== 开始请求 ==========================================
[linkservera] [695ca6b528140ca7802052bea3187450,802052bea3187450] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:49.491] > [INFO] [http-nio-13000-exec-2] - 请求URL: http://localhost:13000/api/common/used
[linkservera] [695ca6b528140ca7802052bea3187450,802052bea3187450] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:49.491] > [INFO] [http-nio-13000-exec-2] - 调用方法: POST
[linkservera] [695ca6b528140ca7802052bea3187450,802052bea3187450] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:49.491] > [INFO] [http-nio-13000-exec-2] - 请求IP: 127.0.0.1
[linkservera] [695ca6b528140ca7802052bea3187450,802052bea3187450] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:49.491] > [INFO] [http-nio-13000-exec-2] - 请求类方法: com.gg.midend.controller.api.CommonApiController.getProcessMemory
[linkservera] [695ca6b528140ca7802052bea3187450,802052bea3187450] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:49.491] > [INFO] [http-nio-13000-exec-2] - 请求参数: {"tradeTime":"2026-03-22 08:18:00","name":"小王","patientId":"123456"}
[linkservera] [695ca6b528140ca7802052bea3187450,802052bea3187450] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:49.541] > [INFO] [http-nio-13000-exec-2] - 耗时: 50毫秒
[linkservera] [695ca6b528140ca7802052bea3187450,802052bea3187450] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:49.541] > [INFO] [http-nio-13000-exec-2] - 返回结果: {"tradeTime":"2026-03-22 08:18:00","name":"小王","patientId":"123456"}
[linkservera] [695ca6b528140ca7802052bea3187450,802052bea3187450] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:49.541] > [INFO] [http-nio-13000-exec-2] - ========================================== 请求结束 ==========================================

bash 复制代码

[linkserverb] [695ca6b346e9e8d63bf96eda44f09c6e,3bf96eda44f09c6e] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:48.309] > [INFO] [http-nio-14000-exec-1] - ========================================== 开始请求 ==========================================
[linkserverb] [695ca6b346e9e8d63bf96eda44f09c6e,3bf96eda44f09c6e] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:48.318] > [INFO] [http-nio-14000-exec-1] - 请求URL: http://localhost:14000/api/common/used
[linkserverb] [695ca6b346e9e8d63bf96eda44f09c6e,3bf96eda44f09c6e] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:48.318] > [INFO] [http-nio-14000-exec-1] - 调用方法: POST
[linkserverb] [695ca6b346e9e8d63bf96eda44f09c6e,3bf96eda44f09c6e] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:48.318] > [INFO] [http-nio-14000-exec-1] - 请求IP: 127.0.0.1
[linkserverb] [695ca6b346e9e8d63bf96eda44f09c6e,3bf96eda44f09c6e] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:48.319] > [INFO] [http-nio-14000-exec-1] - 请求类方法: com.gg.midend.controller.api.CommonApiController.getProcessMemory
[linkserverb] [695ca6b346e9e8d63bf96eda44f09c6e,3bf96eda44f09c6e] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:48.320] > [INFO] [http-nio-14000-exec-1] - 请求参数: {"tradeTime":"2026-03-22 08:18:00","name":"老王","patientId":"123456"}
[linkserverb] [695ca6b346e9e8d63bf96eda44f09c6e,3bf96eda44f09c6e] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:48.955] > [INFO] [http-nio-14000-exec-1] - 耗时: 634毫秒
[linkserverb] [695ca6b346e9e8d63bf96eda44f09c6e,3bf96eda44f09c6e] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:48.956] > [INFO] [http-nio-14000-exec-1] - 返回结果: {"tradeTime":"2026-03-22 08:18:00","name":"老王","patientId":"123456"}
[linkserverb] [695ca6b346e9e8d63bf96eda44f09c6e,3bf96eda44f09c6e] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:48.956] > [INFO] [http-nio-14000-exec-1] - ========================================== 请求结束 ==========================================

[linkserverb] [695ca6b528140ca7802052bea3187450,802052bea3187450] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:48.491] > [INFO] [http-nio-14000-exec-2] - ========================================== 开始请求 ==========================================
[linkserverb] [695ca6b528140ca7802052bea3187450,802052bea3187450] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:48.491] > [INFO] [http-nio-14000-exec-2] - 请求URL: http://localhost:14000/api/common/used
[linkserverb] [695ca6b528140ca7802052bea3187450,802052bea3187450] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:48.491] > [INFO] [http-nio-14000-exec-2] - 调用方法: POST
[linkserverb] [695ca6b528140ca7802052bea3187450,802052bea3187450] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:48.491] > [INFO] [http-nio-14000-exec-2] - 请求IP: 127.0.0.1
[linkserverb] [695ca6b528140ca7802052bea3187450,802052bea3187450] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:48.491] > [INFO] [http-nio-14000-exec-2] - 请求类方法: com.gg.midend.controller.api.CommonApiController.getProcessMemory
[linkserverb] [695ca6b528140ca7802052bea3187450,802052bea3187450] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:48.491] > [INFO] [http-nio-14000-exec-2] - 请求参数: {"tradeTime":"2026-03-22 08:18:00","name":"小王","patientId":"123456"}
[linkserverb] [695ca6b528140ca7802052bea3187450,802052bea3187450] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:48.541] > [INFO] [http-nio-14000-exec-2] - 耗时: 50毫秒
[linkserverb] [695ca6b528140ca7802052bea3187450,802052bea3187450] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:48.541] > [INFO] [http-nio-14000-exec-2] - 返回结果: {"tradeTime":"2026-03-22 08:18:00","name":"小王","patientId":"123456"}
[linkserverb] [695ca6b528140ca7802052bea3187450,802052bea3187450] [com.gg.core.aspect.MethodAspect] [2026-01-06 14:07:48.541] > [INFO] [http-nio-14000-exec-2] - ========================================== 请求结束 ==========================================

3.环境配置

3.1.logstash配置

sql 复制代码

input {
  file {
    path => "/home/geit/midend-center/service3/linkservera/logs/servera.log"
    type => "linkservera"
    start_position => "beginning"
    codec => plain { charset => "UTF-8" }
  }
  file {
    path => "/home/geit/midend-center/service3/linkserverb/logs/serverb.log"
    type => "linkserverb"
    start_position => "beginning"
    codec => plain { charset => "UTF-8" }
  }
}

filter {
  # 1. Grok 解析固定前缀
  grok {
    match => {
      "message" => "\[%{DATA:app}\] \[%{DATA:traceId},%{DATA:spanId}\] \[%{DATA:logger_name}\] \[%{TIMESTAMP_ISO8601:log_time}\] > \[%{LOGLEVEL:level}\] \[%{DATA:thread_name}\] - %{GREEDYDATA:msg_raw}"
    }
  }

  mutate {
    rename => { "msg_raw" => "message" }
  }

  # 2. 解析 "请求参数:" 后面的 JSON 字符串
  if [message] =~ /^请求参数: / {
    grok {
      match => {
        "message" => "^请求参数: %{GREEDYDATA:json_str}"
      }
    }

    json {
      source => "json_str"
      target => "biz_params"
      remove_field => [ "json_str" ]
    }

    # 3. 【关键】将 JSON 里的字段提升到根层级
    # 这样无论 JSON 里有什么字段，都会直接成为 ES 的可搜索字段
    mutate {
      rename => { "[biz_params][tradeTime]" => "tradeTime" }
      rename => { "[biz_params][patientId]" => "patientId" }
      rename => { "[biz_params][name]" => "name" }
      # 如果有其他字段，可以继续加，或者写 ruby 脚本自动遍历 biz_params 提取
      # rename => { "[biz_params][orderId]" => "orderId" }
    }
  }

  # 4. 时间处理
  date {
    match => ["log_time", "yyyy-MM-dd HH:mm:ss.SSS"]
    target => "@timestamp"
  }
  
  mutate {
    remove_field => [ "log_time", "msg_raw" ]
  }
}

output {
  if [type] == "linkservera" or [type] == "linkserverb" {
    elasticsearch {
      hosts => ["http://127.0.0.1:9200"]
      index => "microservice-logs-%{+YYYY.MM.dd}"
    }
  }
}

3.2.设置启动

去除zipkin启动

必须使用geit用户（非root)启动，然后，报错：

bash 复制代码

java.io.FileNotFoundException: /home/geit/midend-center/service3/elasticsearch/logs/geit-es-cluster_server.json (权限不够)

修复：

4.写Java客户端接口

引入依赖：

bash 复制代码

        <!--新增-->
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>2.13.5</version>
        </dependency>

        <!-- Jakarta JSON API（避免 ClassNotFoundException: jakarta.json.spi.JsonProvider） -->
        <dependency>
            <groupId>jakarta.json</groupId>
            <artifactId>jakarta.json-api</artifactId>
            <version>2.0.1</version>
        </dependency>

        <!-- JSON 实现（Parsson） -->
        <dependency>
            <groupId>org.eclipse.parsson</groupId>
            <artifactId>parsson</artifactId>
            <version>1.0.0</version>
        </dependency>

        <!-- Elasticsearch Java API Client 7.17（新版，官方推荐） -->
        <dependency>
            <groupId>co.elastic.clients</groupId>
            <artifactId>elasticsearch-java</artifactId>
            <version>7.17.29</version>
        </dependency>

        <!-- 底层 low-level REST Client -->
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-high-level-client</artifactId>
            <version>7.17.29</version>
        </dependency>

部分代码：

测试：

返回空，查看logstash日志：