系统越简单，日志越像一个附属品，哪里出问题就登录机器看一眼。
系统一旦拆成多个服务，日志就会变成排查问题的入口。订单服务打了一行日志，支付服务打了一行日志，库存服务又打了一行日志，如果这些日志散在不同机器上，排查问题就会变成"到处找文件"。
日志采集要解决的事很直接：把分散在各台机器、各个容器、各个服务里的日志，稳定地收集到统一平台，让开发和运维可以按时间、关键字、服务名、traceId 快速检索和分析。
为什么需要集中日志
单机应用里，日志一般写在本地文件：
text
/data/logs/order-service/info.log
/data/logs/order-service/error.log这在小系统里没问题，常见命令也很好用：
bash
tail -f info.log
grep "orderId=10086" info.log但生产环境通常不是一台机器。
#mermaid-svg-OaZVkf1RkCFPZKH9{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-OaZVkf1RkCFPZKH9 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-OaZVkf1RkCFPZKH9 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-OaZVkf1RkCFPZKH9 .error-icon{fill:#552222;}#mermaid-svg-OaZVkf1RkCFPZKH9 .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-OaZVkf1RkCFPZKH9 .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-OaZVkf1RkCFPZKH9 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-OaZVkf1RkCFPZKH9 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-OaZVkf1RkCFPZKH9 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-OaZVkf1RkCFPZKH9 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-OaZVkf1RkCFPZKH9 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-OaZVkf1RkCFPZKH9 .marker{fill:#333333;stroke:#333333;}#mermaid-svg-OaZVkf1RkCFPZKH9 .marker.cross{stroke:#333333;}#mermaid-svg-OaZVkf1RkCFPZKH9 svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-OaZVkf1RkCFPZKH9 p{margin:0;}#mermaid-svg-OaZVkf1RkCFPZKH9 .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-OaZVkf1RkCFPZKH9 .cluster-label text{fill:#333;}#mermaid-svg-OaZVkf1RkCFPZKH9 .cluster-label span{color:#333;}#mermaid-svg-OaZVkf1RkCFPZKH9 .cluster-label span p{background-color:transparent;}#mermaid-svg-OaZVkf1RkCFPZKH9 .label text,#mermaid-svg-OaZVkf1RkCFPZKH9 span{fill:#333;color:#333;}#mermaid-svg-OaZVkf1RkCFPZKH9 .node rect,#mermaid-svg-OaZVkf1RkCFPZKH9 .node circle,#mermaid-svg-OaZVkf1RkCFPZKH9 .node ellipse,#mermaid-svg-OaZVkf1RkCFPZKH9 .node polygon,#mermaid-svg-OaZVkf1RkCFPZKH9 .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-OaZVkf1RkCFPZKH9 .rough-node .label text,#mermaid-svg-OaZVkf1RkCFPZKH9 .node .label text,#mermaid-svg-OaZVkf1RkCFPZKH9 .image-shape .label,#mermaid-svg-OaZVkf1RkCFPZKH9 .icon-shape .label{text-anchor:middle;}#mermaid-svg-OaZVkf1RkCFPZKH9 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-OaZVkf1RkCFPZKH9 .rough-node .label,#mermaid-svg-OaZVkf1RkCFPZKH9 .node .label,#mermaid-svg-OaZVkf1RkCFPZKH9 .image-shape .label,#mermaid-svg-OaZVkf1RkCFPZKH9 .icon-shape .label{text-align:center;}#mermaid-svg-OaZVkf1RkCFPZKH9 .node.clickable{cursor:pointer;}#mermaid-svg-OaZVkf1RkCFPZKH9 .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-OaZVkf1RkCFPZKH9 .arrowheadPath{fill:#333333;}#mermaid-svg-OaZVkf1RkCFPZKH9 .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-OaZVkf1RkCFPZKH9 .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-OaZVkf1RkCFPZKH9 .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-OaZVkf1RkCFPZKH9 .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-OaZVkf1RkCFPZKH9 .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-OaZVkf1RkCFPZKH9 .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-OaZVkf1RkCFPZKH9 .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-OaZVkf1RkCFPZKH9 .cluster text{fill:#333;}#mermaid-svg-OaZVkf1RkCFPZKH9 .cluster span{color:#333;}#mermaid-svg-OaZVkf1RkCFPZKH9 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-OaZVkf1RkCFPZKH9 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-OaZVkf1RkCFPZKH9 rect.text{fill:none;stroke-width:0;}#mermaid-svg-OaZVkf1RkCFPZKH9 .icon-shape,#mermaid-svg-OaZVkf1RkCFPZKH9 .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-OaZVkf1RkCFPZKH9 .icon-shape p,#mermaid-svg-OaZVkf1RkCFPZKH9 .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-OaZVkf1RkCFPZKH9 .icon-shape .label rect,#mermaid-svg-OaZVkf1RkCFPZKH9 .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-OaZVkf1RkCFPZKH9 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-OaZVkf1RkCFPZKH9 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-OaZVkf1RkCFPZKH9 :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 用户请求
网关
订单服务实例 1
订单服务实例 2
支付服务实例 1
库存服务实例 1
本地日志文件
本地日志文件
本地日志文件
本地日志文件
这时会出现几个很真实的问题：
| 问题 | 影响 |
|---|---|
| 日志分散 | 不知道请求打到哪台机器 |
| 查询低效 | 需要反复登录服务器 |
| 权限混乱 | 给开发开机器权限有风险 |
| 日志易丢 | 容器重启、机器下线可能导致日志不可查 |
| 难以关联 | 跨服务调用很难串起来 |
集中日志平台的价值，不只是把日志"搬到一个地方"。更重要的是，把日志变成可检索、可聚合、可追踪的系统数据。
ELK 分别负责什么
ELK 通常指 Elasticsearch、Logstash、Kibana 三个组件。很多生产环境还会加 Filebeat，所以也常被叫作 Beats + ELK 或 Elastic Stack。
| 组件 | 角色 | 作用 |
|---|---|---|
| Elasticsearch | 存储和检索 | 建索引、按条件搜索日志 |
| Logstash | 处理和转发 | 接收日志、解析字段、过滤转换 |
| Kibana | 查询和展示 | 搜索日志、配置图表、查看趋势 |
如果只讲经典 ELK，一条日志从应用到页面大致是这样走的：
#mermaid-svg-jQc0d5vsS83frvAh{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-jQc0d5vsS83frvAh .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-jQc0d5vsS83frvAh .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-jQc0d5vsS83frvAh .error-icon{fill:#552222;}#mermaid-svg-jQc0d5vsS83frvAh .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-jQc0d5vsS83frvAh .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-jQc0d5vsS83frvAh .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-jQc0d5vsS83frvAh .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-jQc0d5vsS83frvAh .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-jQc0d5vsS83frvAh .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-jQc0d5vsS83frvAh .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-jQc0d5vsS83frvAh .marker{fill:#333333;stroke:#333333;}#mermaid-svg-jQc0d5vsS83frvAh .marker.cross{stroke:#333333;}#mermaid-svg-jQc0d5vsS83frvAh svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-jQc0d5vsS83frvAh p{margin:0;}#mermaid-svg-jQc0d5vsS83frvAh .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-jQc0d5vsS83frvAh .cluster-label text{fill:#333;}#mermaid-svg-jQc0d5vsS83frvAh .cluster-label span{color:#333;}#mermaid-svg-jQc0d5vsS83frvAh .cluster-label span p{background-color:transparent;}#mermaid-svg-jQc0d5vsS83frvAh .label text,#mermaid-svg-jQc0d5vsS83frvAh span{fill:#333;color:#333;}#mermaid-svg-jQc0d5vsS83frvAh .node rect,#mermaid-svg-jQc0d5vsS83frvAh .node circle,#mermaid-svg-jQc0d5vsS83frvAh .node ellipse,#mermaid-svg-jQc0d5vsS83frvAh .node polygon,#mermaid-svg-jQc0d5vsS83frvAh .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-jQc0d5vsS83frvAh .rough-node .label text,#mermaid-svg-jQc0d5vsS83frvAh .node .label text,#mermaid-svg-jQc0d5vsS83frvAh .image-shape .label,#mermaid-svg-jQc0d5vsS83frvAh .icon-shape .label{text-anchor:middle;}#mermaid-svg-jQc0d5vsS83frvAh .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-jQc0d5vsS83frvAh .rough-node .label,#mermaid-svg-jQc0d5vsS83frvAh .node .label,#mermaid-svg-jQc0d5vsS83frvAh .image-shape .label,#mermaid-svg-jQc0d5vsS83frvAh .icon-shape .label{text-align:center;}#mermaid-svg-jQc0d5vsS83frvAh .node.clickable{cursor:pointer;}#mermaid-svg-jQc0d5vsS83frvAh .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-jQc0d5vsS83frvAh .arrowheadPath{fill:#333333;}#mermaid-svg-jQc0d5vsS83frvAh .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-jQc0d5vsS83frvAh .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-jQc0d5vsS83frvAh .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-jQc0d5vsS83frvAh .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-jQc0d5vsS83frvAh .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-jQc0d5vsS83frvAh .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-jQc0d5vsS83frvAh .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-jQc0d5vsS83frvAh .cluster text{fill:#333;}#mermaid-svg-jQc0d5vsS83frvAh .cluster span{color:#333;}#mermaid-svg-jQc0d5vsS83frvAh div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-jQc0d5vsS83frvAh .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-jQc0d5vsS83frvAh rect.text{fill:none;stroke-width:0;}#mermaid-svg-jQc0d5vsS83frvAh .icon-shape,#mermaid-svg-jQc0d5vsS83frvAh .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-jQc0d5vsS83frvAh .icon-shape p,#mermaid-svg-jQc0d5vsS83frvAh .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-jQc0d5vsS83frvAh .icon-shape .label rect,#mermaid-svg-jQc0d5vsS83frvAh .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-jQc0d5vsS83frvAh .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-jQc0d5vsS83frvAh .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-jQc0d5vsS83frvAh :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} Java 应用
日志文件
采集器
Logstash
Elasticsearch
Kibana
检索和分析
但在真实项目里，不太建议让每个 Java 应用都直接推 Logstash。更常见的方式是让 Filebeat 靠近日志文件或容器标准输出，负责轻量采集和转发。Filebeat 是轻量日志转发器，不是复杂处理引擎。
如果日志格式比较简单，也可以让 Filebeat 直接写入 Elasticsearch：
#mermaid-svg-hbmW1obGRHKAqFVQ{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-hbmW1obGRHKAqFVQ .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-hbmW1obGRHKAqFVQ .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-hbmW1obGRHKAqFVQ .error-icon{fill:#552222;}#mermaid-svg-hbmW1obGRHKAqFVQ .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-hbmW1obGRHKAqFVQ .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-hbmW1obGRHKAqFVQ .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-hbmW1obGRHKAqFVQ .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-hbmW1obGRHKAqFVQ .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-hbmW1obGRHKAqFVQ .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-hbmW1obGRHKAqFVQ .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-hbmW1obGRHKAqFVQ .marker{fill:#333333;stroke:#333333;}#mermaid-svg-hbmW1obGRHKAqFVQ .marker.cross{stroke:#333333;}#mermaid-svg-hbmW1obGRHKAqFVQ svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-hbmW1obGRHKAqFVQ p{margin:0;}#mermaid-svg-hbmW1obGRHKAqFVQ .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-hbmW1obGRHKAqFVQ .cluster-label text{fill:#333;}#mermaid-svg-hbmW1obGRHKAqFVQ .cluster-label span{color:#333;}#mermaid-svg-hbmW1obGRHKAqFVQ .cluster-label span p{background-color:transparent;}#mermaid-svg-hbmW1obGRHKAqFVQ .label text,#mermaid-svg-hbmW1obGRHKAqFVQ span{fill:#333;color:#333;}#mermaid-svg-hbmW1obGRHKAqFVQ .node rect,#mermaid-svg-hbmW1obGRHKAqFVQ .node circle,#mermaid-svg-hbmW1obGRHKAqFVQ .node ellipse,#mermaid-svg-hbmW1obGRHKAqFVQ .node polygon,#mermaid-svg-hbmW1obGRHKAqFVQ .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-hbmW1obGRHKAqFVQ .rough-node .label text,#mermaid-svg-hbmW1obGRHKAqFVQ .node .label text,#mermaid-svg-hbmW1obGRHKAqFVQ .image-shape .label,#mermaid-svg-hbmW1obGRHKAqFVQ .icon-shape .label{text-anchor:middle;}#mermaid-svg-hbmW1obGRHKAqFVQ .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-hbmW1obGRHKAqFVQ .rough-node .label,#mermaid-svg-hbmW1obGRHKAqFVQ .node .label,#mermaid-svg-hbmW1obGRHKAqFVQ .image-shape .label,#mermaid-svg-hbmW1obGRHKAqFVQ .icon-shape .label{text-align:center;}#mermaid-svg-hbmW1obGRHKAqFVQ .node.clickable{cursor:pointer;}#mermaid-svg-hbmW1obGRHKAqFVQ .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-hbmW1obGRHKAqFVQ .arrowheadPath{fill:#333333;}#mermaid-svg-hbmW1obGRHKAqFVQ .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-hbmW1obGRHKAqFVQ .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-hbmW1obGRHKAqFVQ .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-hbmW1obGRHKAqFVQ .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-hbmW1obGRHKAqFVQ .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-hbmW1obGRHKAqFVQ .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-hbmW1obGRHKAqFVQ .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-hbmW1obGRHKAqFVQ .cluster text{fill:#333;}#mermaid-svg-hbmW1obGRHKAqFVQ .cluster span{color:#333;}#mermaid-svg-hbmW1obGRHKAqFVQ div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-hbmW1obGRHKAqFVQ .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-hbmW1obGRHKAqFVQ rect.text{fill:none;stroke-width:0;}#mermaid-svg-hbmW1obGRHKAqFVQ .icon-shape,#mermaid-svg-hbmW1obGRHKAqFVQ .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-hbmW1obGRHKAqFVQ .icon-shape p,#mermaid-svg-hbmW1obGRHKAqFVQ .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-hbmW1obGRHKAqFVQ .icon-shape .label rect,#mermaid-svg-hbmW1obGRHKAqFVQ .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-hbmW1obGRHKAqFVQ .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-hbmW1obGRHKAqFVQ .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-hbmW1obGRHKAqFVQ :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 应用日志
Filebeat
Elasticsearch
Kibana
如果日志需要复杂解析、清洗、字段转换、路由分发，就把 Logstash 放在中间：
#mermaid-svg-oTqjtO4PlCyrCJm1{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-oTqjtO4PlCyrCJm1 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-oTqjtO4PlCyrCJm1 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-oTqjtO4PlCyrCJm1 .error-icon{fill:#552222;}#mermaid-svg-oTqjtO4PlCyrCJm1 .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-oTqjtO4PlCyrCJm1 .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-oTqjtO4PlCyrCJm1 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-oTqjtO4PlCyrCJm1 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-oTqjtO4PlCyrCJm1 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-oTqjtO4PlCyrCJm1 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-oTqjtO4PlCyrCJm1 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-oTqjtO4PlCyrCJm1 .marker{fill:#333333;stroke:#333333;}#mermaid-svg-oTqjtO4PlCyrCJm1 .marker.cross{stroke:#333333;}#mermaid-svg-oTqjtO4PlCyrCJm1 svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-oTqjtO4PlCyrCJm1 p{margin:0;}#mermaid-svg-oTqjtO4PlCyrCJm1 .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-oTqjtO4PlCyrCJm1 .cluster-label text{fill:#333;}#mermaid-svg-oTqjtO4PlCyrCJm1 .cluster-label span{color:#333;}#mermaid-svg-oTqjtO4PlCyrCJm1 .cluster-label span p{background-color:transparent;}#mermaid-svg-oTqjtO4PlCyrCJm1 .label text,#mermaid-svg-oTqjtO4PlCyrCJm1 span{fill:#333;color:#333;}#mermaid-svg-oTqjtO4PlCyrCJm1 .node rect,#mermaid-svg-oTqjtO4PlCyrCJm1 .node circle,#mermaid-svg-oTqjtO4PlCyrCJm1 .node ellipse,#mermaid-svg-oTqjtO4PlCyrCJm1 .node polygon,#mermaid-svg-oTqjtO4PlCyrCJm1 .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-oTqjtO4PlCyrCJm1 .rough-node .label text,#mermaid-svg-oTqjtO4PlCyrCJm1 .node .label text,#mermaid-svg-oTqjtO4PlCyrCJm1 .image-shape .label,#mermaid-svg-oTqjtO4PlCyrCJm1 .icon-shape .label{text-anchor:middle;}#mermaid-svg-oTqjtO4PlCyrCJm1 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-oTqjtO4PlCyrCJm1 .rough-node .label,#mermaid-svg-oTqjtO4PlCyrCJm1 .node .label,#mermaid-svg-oTqjtO4PlCyrCJm1 .image-shape .label,#mermaid-svg-oTqjtO4PlCyrCJm1 .icon-shape .label{text-align:center;}#mermaid-svg-oTqjtO4PlCyrCJm1 .node.clickable{cursor:pointer;}#mermaid-svg-oTqjtO4PlCyrCJm1 .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-oTqjtO4PlCyrCJm1 .arrowheadPath{fill:#333333;}#mermaid-svg-oTqjtO4PlCyrCJm1 .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-oTqjtO4PlCyrCJm1 .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-oTqjtO4PlCyrCJm1 .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-oTqjtO4PlCyrCJm1 .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-oTqjtO4PlCyrCJm1 .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-oTqjtO4PlCyrCJm1 .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-oTqjtO4PlCyrCJm1 .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-oTqjtO4PlCyrCJm1 .cluster text{fill:#333;}#mermaid-svg-oTqjtO4PlCyrCJm1 .cluster span{color:#333;}#mermaid-svg-oTqjtO4PlCyrCJm1 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-oTqjtO4PlCyrCJm1 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-oTqjtO4PlCyrCJm1 rect.text{fill:none;stroke-width:0;}#mermaid-svg-oTqjtO4PlCyrCJm1 .icon-shape,#mermaid-svg-oTqjtO4PlCyrCJm1 .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-oTqjtO4PlCyrCJm1 .icon-shape p,#mermaid-svg-oTqjtO4PlCyrCJm1 .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-oTqjtO4PlCyrCJm1 .icon-shape .label rect,#mermaid-svg-oTqjtO4PlCyrCJm1 .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-oTqjtO4PlCyrCJm1 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-oTqjtO4PlCyrCJm1 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-oTqjtO4PlCyrCJm1 :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 应用日志
Filebeat
Logstash
字段解析
过滤清洗
Elasticsearch
Kibana
实际项目里更常见的是 Filebeat 加 Logstash 加 Elasticsearch 加 Kibana。Filebeat 靠近业务机器，负责轻量采集；Logstash 放在日志链路中间，负责重一点的处理逻辑。这样应用只管把日志写好，采集链路单独演进，耦合更低。
Filebeat 和 Logstash 怎么选
很多人第一次搭日志平台，会把 Filebeat 和 Logstash 的职责混在一起。
Filebeat 更像一个轻量日志搬运工。它部署在业务服务器或容器节点上，监控指定日志文件，把新增日志发送出去。它本身比较轻，不适合承担太复杂的处理逻辑。
Logstash 更像一个日志加工管道。它可以接收多种输入，经过 filter 做解析、转换、补充字段，再输出到 Elasticsearch、Kafka 等地方。
#mermaid-svg-Dyehe3uBO7CfGEyd{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-Dyehe3uBO7CfGEyd .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-Dyehe3uBO7CfGEyd .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-Dyehe3uBO7CfGEyd .error-icon{fill:#552222;}#mermaid-svg-Dyehe3uBO7CfGEyd .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-Dyehe3uBO7CfGEyd .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-Dyehe3uBO7CfGEyd .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-Dyehe3uBO7CfGEyd .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-Dyehe3uBO7CfGEyd .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-Dyehe3uBO7CfGEyd .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-Dyehe3uBO7CfGEyd .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-Dyehe3uBO7CfGEyd .marker{fill:#333333;stroke:#333333;}#mermaid-svg-Dyehe3uBO7CfGEyd .marker.cross{stroke:#333333;}#mermaid-svg-Dyehe3uBO7CfGEyd svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-Dyehe3uBO7CfGEyd p{margin:0;}#mermaid-svg-Dyehe3uBO7CfGEyd .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-Dyehe3uBO7CfGEyd .cluster-label text{fill:#333;}#mermaid-svg-Dyehe3uBO7CfGEyd .cluster-label span{color:#333;}#mermaid-svg-Dyehe3uBO7CfGEyd .cluster-label span p{background-color:transparent;}#mermaid-svg-Dyehe3uBO7CfGEyd .label text,#mermaid-svg-Dyehe3uBO7CfGEyd span{fill:#333;color:#333;}#mermaid-svg-Dyehe3uBO7CfGEyd .node rect,#mermaid-svg-Dyehe3uBO7CfGEyd .node circle,#mermaid-svg-Dyehe3uBO7CfGEyd .node ellipse,#mermaid-svg-Dyehe3uBO7CfGEyd .node polygon,#mermaid-svg-Dyehe3uBO7CfGEyd .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-Dyehe3uBO7CfGEyd .rough-node .label text,#mermaid-svg-Dyehe3uBO7CfGEyd .node .label text,#mermaid-svg-Dyehe3uBO7CfGEyd .image-shape .label,#mermaid-svg-Dyehe3uBO7CfGEyd .icon-shape .label{text-anchor:middle;}#mermaid-svg-Dyehe3uBO7CfGEyd .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-Dyehe3uBO7CfGEyd .rough-node .label,#mermaid-svg-Dyehe3uBO7CfGEyd .node .label,#mermaid-svg-Dyehe3uBO7CfGEyd .image-shape .label,#mermaid-svg-Dyehe3uBO7CfGEyd .icon-shape .label{text-align:center;}#mermaid-svg-Dyehe3uBO7CfGEyd .node.clickable{cursor:pointer;}#mermaid-svg-Dyehe3uBO7CfGEyd .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-Dyehe3uBO7CfGEyd .arrowheadPath{fill:#333333;}#mermaid-svg-Dyehe3uBO7CfGEyd .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-Dyehe3uBO7CfGEyd .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-Dyehe3uBO7CfGEyd .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-Dyehe3uBO7CfGEyd .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-Dyehe3uBO7CfGEyd .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-Dyehe3uBO7CfGEyd .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-Dyehe3uBO7CfGEyd .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-Dyehe3uBO7CfGEyd .cluster text{fill:#333;}#mermaid-svg-Dyehe3uBO7CfGEyd .cluster span{color:#333;}#mermaid-svg-Dyehe3uBO7CfGEyd div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-Dyehe3uBO7CfGEyd .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-Dyehe3uBO7CfGEyd rect.text{fill:none;stroke-width:0;}#mermaid-svg-Dyehe3uBO7CfGEyd .icon-shape,#mermaid-svg-Dyehe3uBO7CfGEyd .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-Dyehe3uBO7CfGEyd .icon-shape p,#mermaid-svg-Dyehe3uBO7CfGEyd .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-Dyehe3uBO7CfGEyd .icon-shape .label rect,#mermaid-svg-Dyehe3uBO7CfGEyd .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-Dyehe3uBO7CfGEyd .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-Dyehe3uBO7CfGEyd .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-Dyehe3uBO7CfGEyd :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 不需要
需要
日志采集需求
是否需要复杂处理
Filebeat 直接写入 Elasticsearch
Filebeat 写入 Logstash
Logstash 解析和过滤
写入 Elasticsearch
可以用一个简单判断：
| 场景 | 建议 |
|---|---|
| 只采集普通文本日志 | Filebeat 直连 Elasticsearch |
| 需要解析 JSON 字段 | Filebeat 或 Logstash 都可以 |
| 需要多条件过滤和转换 | 加 Logstash |
| 需要缓冲削峰 | 中间可接 Kafka |
| 日志来源很多且格式复杂 | Logstash 更合适 |
高并发系统里，还会在采集链路中间加 Kafka。
#mermaid-svg-S1sTx4hZTDGxN9VC{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-S1sTx4hZTDGxN9VC .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-S1sTx4hZTDGxN9VC .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-S1sTx4hZTDGxN9VC .error-icon{fill:#552222;}#mermaid-svg-S1sTx4hZTDGxN9VC .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-S1sTx4hZTDGxN9VC .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-S1sTx4hZTDGxN9VC .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-S1sTx4hZTDGxN9VC .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-S1sTx4hZTDGxN9VC .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-S1sTx4hZTDGxN9VC .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-S1sTx4hZTDGxN9VC .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-S1sTx4hZTDGxN9VC .marker{fill:#333333;stroke:#333333;}#mermaid-svg-S1sTx4hZTDGxN9VC .marker.cross{stroke:#333333;}#mermaid-svg-S1sTx4hZTDGxN9VC svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-S1sTx4hZTDGxN9VC p{margin:0;}#mermaid-svg-S1sTx4hZTDGxN9VC .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-S1sTx4hZTDGxN9VC .cluster-label text{fill:#333;}#mermaid-svg-S1sTx4hZTDGxN9VC .cluster-label span{color:#333;}#mermaid-svg-S1sTx4hZTDGxN9VC .cluster-label span p{background-color:transparent;}#mermaid-svg-S1sTx4hZTDGxN9VC .label text,#mermaid-svg-S1sTx4hZTDGxN9VC span{fill:#333;color:#333;}#mermaid-svg-S1sTx4hZTDGxN9VC .node rect,#mermaid-svg-S1sTx4hZTDGxN9VC .node circle,#mermaid-svg-S1sTx4hZTDGxN9VC .node ellipse,#mermaid-svg-S1sTx4hZTDGxN9VC .node polygon,#mermaid-svg-S1sTx4hZTDGxN9VC .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-S1sTx4hZTDGxN9VC .rough-node .label text,#mermaid-svg-S1sTx4hZTDGxN9VC .node .label text,#mermaid-svg-S1sTx4hZTDGxN9VC .image-shape .label,#mermaid-svg-S1sTx4hZTDGxN9VC .icon-shape .label{text-anchor:middle;}#mermaid-svg-S1sTx4hZTDGxN9VC .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-S1sTx4hZTDGxN9VC .rough-node .label,#mermaid-svg-S1sTx4hZTDGxN9VC .node .label,#mermaid-svg-S1sTx4hZTDGxN9VC .image-shape .label,#mermaid-svg-S1sTx4hZTDGxN9VC .icon-shape .label{text-align:center;}#mermaid-svg-S1sTx4hZTDGxN9VC .node.clickable{cursor:pointer;}#mermaid-svg-S1sTx4hZTDGxN9VC .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-S1sTx4hZTDGxN9VC .arrowheadPath{fill:#333333;}#mermaid-svg-S1sTx4hZTDGxN9VC .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-S1sTx4hZTDGxN9VC .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-S1sTx4hZTDGxN9VC .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-S1sTx4hZTDGxN9VC .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-S1sTx4hZTDGxN9VC .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-S1sTx4hZTDGxN9VC .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-S1sTx4hZTDGxN9VC .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-S1sTx4hZTDGxN9VC .cluster text{fill:#333;}#mermaid-svg-S1sTx4hZTDGxN9VC .cluster span{color:#333;}#mermaid-svg-S1sTx4hZTDGxN9VC div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-S1sTx4hZTDGxN9VC .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-S1sTx4hZTDGxN9VC rect.text{fill:none;stroke-width:0;}#mermaid-svg-S1sTx4hZTDGxN9VC .icon-shape,#mermaid-svg-S1sTx4hZTDGxN9VC .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-S1sTx4hZTDGxN9VC .icon-shape p,#mermaid-svg-S1sTx4hZTDGxN9VC .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-S1sTx4hZTDGxN9VC .icon-shape .label rect,#mermaid-svg-S1sTx4hZTDGxN9VC .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-S1sTx4hZTDGxN9VC .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-S1sTx4hZTDGxN9VC .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-S1sTx4hZTDGxN9VC :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 业务应用
Filebeat
Kafka
Logstash
Elasticsearch
Kibana
Kafka 的作用不是必须的，但它能把"日志产生速度"和"日志处理速度"解耦。短时间日志暴涨时，先写入 Kafka，后面的 Logstash 慢慢消费，日志链路会更稳。代价是链路更长，运维成本更高，所以中小系统不必一开始就上 Kafka。
应用日志应该怎么打
日志平台好不好用，一半取决于采集链路，另一半取决于应用写日志的质量。
最怕的是这种日志：
text
下单失败它对排查几乎没有帮助。失败的是哪个订单，哪个用户，哪个接口，失败原因是什么，都看不出来。
更好的日志应该带上关键上下文：
text
create order failed, orderId=10086, userId=9527, skuId=3001, reason=stock_not_enough如果是 Java 项目，常见组合是 Logback 加 JSON 编码器，把日志输出成结构化 JSON。这样 Elasticsearch 能直接按字段检索，而不是只做全文搜索。
json
{
"timestamp": "2026-06-08T10:15:30.123+08:00",
"level": "ERROR",
"service": "order-service",
"traceId": "a1b2c3d4",
"spanId": "s1001",
"userId": 9527,
"orderId": 10086,
"message": "create order failed",
"exception": "StockNotEnoughException"
}结构化日志的好处很明显：
| 字段 | 用途 |
|---|---|
timestamp |
按时间定位问题 |
level |
区分 INFO、WARN、ERROR |
service |
知道日志来自哪个服务 |
traceId |
串起一次请求的完整链路 |
userId |
定位用户维度的问题 |
orderId |
定位业务对象 |
exception |
快速聚合异常类型 |
日志不是写得越多越好，而是要让后来排查问题的人知道"发生了什么、发生在哪、影响谁、为什么失败"。
traceId 是日志检索的生命线
微服务排查问题时，最重要的字段之一就是 traceId。
一次请求可能经过网关、订单、库存、优惠券、支付多个服务。如果没有统一标识，每个服务的日志都像孤立的小片段。
"支付服务" "库存服务" "订单服务" "网关" "用户" "支付服务" "库存服务" "订单服务" "网关" "用户" #mermaid-svg-dohiZytz3kfhkuHg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-dohiZytz3kfhkuHg .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-dohiZytz3kfhkuHg .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-dohiZytz3kfhkuHg .error-icon{fill:#552222;}#mermaid-svg-dohiZytz3kfhkuHg .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-dohiZytz3kfhkuHg .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-dohiZytz3kfhkuHg .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-dohiZytz3kfhkuHg .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-dohiZytz3kfhkuHg .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-dohiZytz3kfhkuHg .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-dohiZytz3kfhkuHg .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-dohiZytz3kfhkuHg .marker{fill:#333333;stroke:#333333;}#mermaid-svg-dohiZytz3kfhkuHg .marker.cross{stroke:#333333;}#mermaid-svg-dohiZytz3kfhkuHg svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-dohiZytz3kfhkuHg p{margin:0;}#mermaid-svg-dohiZytz3kfhkuHg .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-dohiZytz3kfhkuHg text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-dohiZytz3kfhkuHg .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-dohiZytz3kfhkuHg .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-dohiZytz3kfhkuHg .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-dohiZytz3kfhkuHg .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-dohiZytz3kfhkuHg #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-dohiZytz3kfhkuHg .sequenceNumber{fill:white;}#mermaid-svg-dohiZytz3kfhkuHg #sequencenumber{fill:#333;}#mermaid-svg-dohiZytz3kfhkuHg #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-dohiZytz3kfhkuHg .messageText{fill:#333;stroke:none;}#mermaid-svg-dohiZytz3kfhkuHg .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-dohiZytz3kfhkuHg .labelText,#mermaid-svg-dohiZytz3kfhkuHg .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-dohiZytz3kfhkuHg .loopText,#mermaid-svg-dohiZytz3kfhkuHg .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-dohiZytz3kfhkuHg .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-dohiZytz3kfhkuHg .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-dohiZytz3kfhkuHg .noteText,#mermaid-svg-dohiZytz3kfhkuHg .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-dohiZytz3kfhkuHg .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-dohiZytz3kfhkuHg .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-dohiZytz3kfhkuHg .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-dohiZytz3kfhkuHg .actorPopupMenu{position:absolute;}#mermaid-svg-dohiZytz3kfhkuHg .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-dohiZytz3kfhkuHg .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-dohiZytz3kfhkuHg .actor-man circle,#mermaid-svg-dohiZytz3kfhkuHg line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-dohiZytz3kfhkuHg :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} "提交下单请求""携带 traceId""携带 traceId""携带 traceId""返回库存结果""返回支付结果""返回下单结果""响应"
有了 traceId，排查时只要在 Kibana 里搜索：
text
traceId:a1b2c3d4就能看到这次请求经过的所有服务日志。
如果系统还接了链路追踪，traceId 还可以和调用链详情关联起来，日志负责告诉你"哪里报错"，链路追踪负责告诉你"哪里慢、调用顺序是什么"。
本地日志命令仍然要会
有了集中日志平台，不代表本地命令就没用了。
当日志采集延迟、平台不可用、临时排查机器问题时，Linux 日志命令仍然非常重要。
查看最新日志：
bash
tail -f app.log查看最后 200 行：
bash
tail -n 200 app.log查看文件开头：
bash
head -n 50 app.log按关键字搜索：
bash
grep "orderId=10086" app.log忽略大小写搜索：
bash
grep -i "exception" app.log查看某个时间段附近的日志：
bash
sed -n '1000,1100p' app.log分页查看：
bash
more app.log把搜索结果输出到新文件：
bash
grep "ERROR" app.log > error.log追加输出：
bash
grep "timeout" app.log >> timeout.log这些命令看起来基础，但生产排查时非常管用。尤其是 tail -f、grep、sed，几乎是日志排查三件套。
日志采集链路的关键配置
日志采集不是把组件装起来就结束了，还要关注几个关键点。
日志路径
采集器必须知道日志在哪里。
yaml
filebeat.inputs:
- type: filestream
id: order-service-log
paths:
- /data/logs/order-service/*.log容器环境里，日志路径可能来自宿主机挂载目录，也可能来自容器标准输出。不同部署方式要提前定好日志规范，否则后面会非常乱。
多行日志
Java 异常堆栈通常是多行。
text
java.lang.RuntimeException: create order failed
at com.example.OrderService.create(OrderService.java:35)
at com.example.OrderController.create(OrderController.java:18)如果不处理多行合并，异常堆栈会被拆成多条日志，检索体验很差。
#mermaid-svg-xCZUKPLBNM6rPe1A{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-xCZUKPLBNM6rPe1A .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-xCZUKPLBNM6rPe1A .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-xCZUKPLBNM6rPe1A .error-icon{fill:#552222;}#mermaid-svg-xCZUKPLBNM6rPe1A .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-xCZUKPLBNM6rPe1A .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-xCZUKPLBNM6rPe1A .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-xCZUKPLBNM6rPe1A .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-xCZUKPLBNM6rPe1A .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-xCZUKPLBNM6rPe1A .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-xCZUKPLBNM6rPe1A .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-xCZUKPLBNM6rPe1A .marker{fill:#333333;stroke:#333333;}#mermaid-svg-xCZUKPLBNM6rPe1A .marker.cross{stroke:#333333;}#mermaid-svg-xCZUKPLBNM6rPe1A svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-xCZUKPLBNM6rPe1A p{margin:0;}#mermaid-svg-xCZUKPLBNM6rPe1A .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-xCZUKPLBNM6rPe1A .cluster-label text{fill:#333;}#mermaid-svg-xCZUKPLBNM6rPe1A .cluster-label span{color:#333;}#mermaid-svg-xCZUKPLBNM6rPe1A .cluster-label span p{background-color:transparent;}#mermaid-svg-xCZUKPLBNM6rPe1A .label text,#mermaid-svg-xCZUKPLBNM6rPe1A span{fill:#333;color:#333;}#mermaid-svg-xCZUKPLBNM6rPe1A .node rect,#mermaid-svg-xCZUKPLBNM6rPe1A .node circle,#mermaid-svg-xCZUKPLBNM6rPe1A .node ellipse,#mermaid-svg-xCZUKPLBNM6rPe1A .node polygon,#mermaid-svg-xCZUKPLBNM6rPe1A .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-xCZUKPLBNM6rPe1A .rough-node .label text,#mermaid-svg-xCZUKPLBNM6rPe1A .node .label text,#mermaid-svg-xCZUKPLBNM6rPe1A .image-shape .label,#mermaid-svg-xCZUKPLBNM6rPe1A .icon-shape .label{text-anchor:middle;}#mermaid-svg-xCZUKPLBNM6rPe1A .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-xCZUKPLBNM6rPe1A .rough-node .label,#mermaid-svg-xCZUKPLBNM6rPe1A .node .label,#mermaid-svg-xCZUKPLBNM6rPe1A .image-shape .label,#mermaid-svg-xCZUKPLBNM6rPe1A .icon-shape .label{text-align:center;}#mermaid-svg-xCZUKPLBNM6rPe1A .node.clickable{cursor:pointer;}#mermaid-svg-xCZUKPLBNM6rPe1A .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-xCZUKPLBNM6rPe1A .arrowheadPath{fill:#333333;}#mermaid-svg-xCZUKPLBNM6rPe1A .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-xCZUKPLBNM6rPe1A .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-xCZUKPLBNM6rPe1A .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-xCZUKPLBNM6rPe1A .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-xCZUKPLBNM6rPe1A .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-xCZUKPLBNM6rPe1A .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-xCZUKPLBNM6rPe1A .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-xCZUKPLBNM6rPe1A .cluster text{fill:#333;}#mermaid-svg-xCZUKPLBNM6rPe1A .cluster span{color:#333;}#mermaid-svg-xCZUKPLBNM6rPe1A div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-xCZUKPLBNM6rPe1A .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-xCZUKPLBNM6rPe1A rect.text{fill:none;stroke-width:0;}#mermaid-svg-xCZUKPLBNM6rPe1A .icon-shape,#mermaid-svg-xCZUKPLBNM6rPe1A .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-xCZUKPLBNM6rPe1A .icon-shape p,#mermaid-svg-xCZUKPLBNM6rPe1A .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-xCZUKPLBNM6rPe1A .icon-shape .label rect,#mermaid-svg-xCZUKPLBNM6rPe1A .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-xCZUKPLBNM6rPe1A .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-xCZUKPLBNM6rPe1A .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-xCZUKPLBNM6rPe1A :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 未合并
已合并
异常堆栈
是否合并多行
多条碎片日志
一条完整异常日志
方便检索和定位
索引规划
Elasticsearch 里日志一般按服务、环境、日期拆索引。常见命名类似：
text
log-order-service-prod-2026.06.08这样做有几个好处：
| 设计 | 好处 |
|---|---|
| 按环境区分 | 避免测试和生产混在一起 |
| 按服务区分 | 查询范围更小 |
| 按日期区分 | 方便冷热数据管理 |
| 设置保留周期 | 控制存储成本 |
日志量大时，索引生命周期管理很重要。不是所有日志都需要永久保存，通常会按业务重要性保留 7 天、30 天、90 天或更久。
字段脱敏
日志里不要打印密码、身份证号、银行卡号、Token、密钥。
这不是"最好不要"，而是必须避免。
#mermaid-svg-XKiscv0kCfoiAGNo{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-XKiscv0kCfoiAGNo .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-XKiscv0kCfoiAGNo .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-XKiscv0kCfoiAGNo .error-icon{fill:#552222;}#mermaid-svg-XKiscv0kCfoiAGNo .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-XKiscv0kCfoiAGNo .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-XKiscv0kCfoiAGNo .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-XKiscv0kCfoiAGNo .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-XKiscv0kCfoiAGNo .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-XKiscv0kCfoiAGNo .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-XKiscv0kCfoiAGNo .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-XKiscv0kCfoiAGNo .marker{fill:#333333;stroke:#333333;}#mermaid-svg-XKiscv0kCfoiAGNo .marker.cross{stroke:#333333;}#mermaid-svg-XKiscv0kCfoiAGNo svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-XKiscv0kCfoiAGNo p{margin:0;}#mermaid-svg-XKiscv0kCfoiAGNo .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-XKiscv0kCfoiAGNo .cluster-label text{fill:#333;}#mermaid-svg-XKiscv0kCfoiAGNo .cluster-label span{color:#333;}#mermaid-svg-XKiscv0kCfoiAGNo .cluster-label span p{background-color:transparent;}#mermaid-svg-XKiscv0kCfoiAGNo .label text,#mermaid-svg-XKiscv0kCfoiAGNo span{fill:#333;color:#333;}#mermaid-svg-XKiscv0kCfoiAGNo .node rect,#mermaid-svg-XKiscv0kCfoiAGNo .node circle,#mermaid-svg-XKiscv0kCfoiAGNo .node ellipse,#mermaid-svg-XKiscv0kCfoiAGNo .node polygon,#mermaid-svg-XKiscv0kCfoiAGNo .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-XKiscv0kCfoiAGNo .rough-node .label text,#mermaid-svg-XKiscv0kCfoiAGNo .node .label text,#mermaid-svg-XKiscv0kCfoiAGNo .image-shape .label,#mermaid-svg-XKiscv0kCfoiAGNo .icon-shape .label{text-anchor:middle;}#mermaid-svg-XKiscv0kCfoiAGNo .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-XKiscv0kCfoiAGNo .rough-node .label,#mermaid-svg-XKiscv0kCfoiAGNo .node .label,#mermaid-svg-XKiscv0kCfoiAGNo .image-shape .label,#mermaid-svg-XKiscv0kCfoiAGNo .icon-shape .label{text-align:center;}#mermaid-svg-XKiscv0kCfoiAGNo .node.clickable{cursor:pointer;}#mermaid-svg-XKiscv0kCfoiAGNo .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-XKiscv0kCfoiAGNo .arrowheadPath{fill:#333333;}#mermaid-svg-XKiscv0kCfoiAGNo .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-XKiscv0kCfoiAGNo .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-XKiscv0kCfoiAGNo .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-XKiscv0kCfoiAGNo .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-XKiscv0kCfoiAGNo .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-XKiscv0kCfoiAGNo .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-XKiscv0kCfoiAGNo .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-XKiscv0kCfoiAGNo .cluster text{fill:#333;}#mermaid-svg-XKiscv0kCfoiAGNo .cluster span{color:#333;}#mermaid-svg-XKiscv0kCfoiAGNo div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-XKiscv0kCfoiAGNo .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-XKiscv0kCfoiAGNo rect.text{fill:none;stroke-width:0;}#mermaid-svg-XKiscv0kCfoiAGNo .icon-shape,#mermaid-svg-XKiscv0kCfoiAGNo .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-XKiscv0kCfoiAGNo .icon-shape p,#mermaid-svg-XKiscv0kCfoiAGNo .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-XKiscv0kCfoiAGNo .icon-shape .label rect,#mermaid-svg-XKiscv0kCfoiAGNo .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-XKiscv0kCfoiAGNo .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-XKiscv0kCfoiAGNo .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-XKiscv0kCfoiAGNo :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 是
否
业务日志
是否包含敏感字段
脱敏或不打印
正常输出
进入采集链路
常见做法是：
| 类型 | 处理方式 |
|---|---|
| 手机号 | 保留前三后四 |
| 身份证号 | 只保留少量可定位片段 |
| 密码 | 禁止打印 |
| Token | 禁止打印或只打印摘要 |
| 请求体 | 敏感接口不要完整打印 |
日志一旦进入集中平台，访问面会变大。前面不脱敏，后面再补权限控制，成本会高很多。
一套比较稳的落地方案
中小规模系统可以这样设计：
#mermaid-svg-uluGFr3ImoQj7etZ{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-uluGFr3ImoQj7etZ .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-uluGFr3ImoQj7etZ .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-uluGFr3ImoQj7etZ .error-icon{fill:#552222;}#mermaid-svg-uluGFr3ImoQj7etZ .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-uluGFr3ImoQj7etZ .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-uluGFr3ImoQj7etZ .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-uluGFr3ImoQj7etZ .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-uluGFr3ImoQj7etZ .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-uluGFr3ImoQj7etZ .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-uluGFr3ImoQj7etZ .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-uluGFr3ImoQj7etZ .marker{fill:#333333;stroke:#333333;}#mermaid-svg-uluGFr3ImoQj7etZ .marker.cross{stroke:#333333;}#mermaid-svg-uluGFr3ImoQj7etZ svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-uluGFr3ImoQj7etZ p{margin:0;}#mermaid-svg-uluGFr3ImoQj7etZ .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-uluGFr3ImoQj7etZ .cluster-label text{fill:#333;}#mermaid-svg-uluGFr3ImoQj7etZ .cluster-label span{color:#333;}#mermaid-svg-uluGFr3ImoQj7etZ .cluster-label span p{background-color:transparent;}#mermaid-svg-uluGFr3ImoQj7etZ .label text,#mermaid-svg-uluGFr3ImoQj7etZ span{fill:#333;color:#333;}#mermaid-svg-uluGFr3ImoQj7etZ .node rect,#mermaid-svg-uluGFr3ImoQj7etZ .node circle,#mermaid-svg-uluGFr3ImoQj7etZ .node ellipse,#mermaid-svg-uluGFr3ImoQj7etZ .node polygon,#mermaid-svg-uluGFr3ImoQj7etZ .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-uluGFr3ImoQj7etZ .rough-node .label text,#mermaid-svg-uluGFr3ImoQj7etZ .node .label text,#mermaid-svg-uluGFr3ImoQj7etZ .image-shape .label,#mermaid-svg-uluGFr3ImoQj7etZ .icon-shape .label{text-anchor:middle;}#mermaid-svg-uluGFr3ImoQj7etZ .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-uluGFr3ImoQj7etZ .rough-node .label,#mermaid-svg-uluGFr3ImoQj7etZ .node .label,#mermaid-svg-uluGFr3ImoQj7etZ .image-shape .label,#mermaid-svg-uluGFr3ImoQj7etZ .icon-shape .label{text-align:center;}#mermaid-svg-uluGFr3ImoQj7etZ .node.clickable{cursor:pointer;}#mermaid-svg-uluGFr3ImoQj7etZ .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-uluGFr3ImoQj7etZ .arrowheadPath{fill:#333333;}#mermaid-svg-uluGFr3ImoQj7etZ .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-uluGFr3ImoQj7etZ .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-uluGFr3ImoQj7etZ .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-uluGFr3ImoQj7etZ .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-uluGFr3ImoQj7etZ .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-uluGFr3ImoQj7etZ .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-uluGFr3ImoQj7etZ .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-uluGFr3ImoQj7etZ .cluster text{fill:#333;}#mermaid-svg-uluGFr3ImoQj7etZ .cluster span{color:#333;}#mermaid-svg-uluGFr3ImoQj7etZ div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-uluGFr3ImoQj7etZ .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-uluGFr3ImoQj7etZ rect.text{fill:none;stroke-width:0;}#mermaid-svg-uluGFr3ImoQj7etZ .icon-shape,#mermaid-svg-uluGFr3ImoQj7etZ .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-uluGFr3ImoQj7etZ .icon-shape p,#mermaid-svg-uluGFr3ImoQj7etZ .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-uluGFr3ImoQj7etZ .icon-shape .label rect,#mermaid-svg-uluGFr3ImoQj7etZ .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-uluGFr3ImoQj7etZ .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-uluGFr3ImoQj7etZ .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-uluGFr3ImoQj7etZ :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} Java 服务
Logback 输出 JSON 日志
本地日志文件
Filebeat 采集
Logstash 解析和过滤
Elasticsearch 存储索引
Kibana 查询分析
网关或拦截器
生成 traceId
更高并发、日志量更大的系统，可以把 Kafka 放进来：
#mermaid-svg-AjpIYA3CwdD0N1Xw{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-AjpIYA3CwdD0N1Xw .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-AjpIYA3CwdD0N1Xw .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-AjpIYA3CwdD0N1Xw .error-icon{fill:#552222;}#mermaid-svg-AjpIYA3CwdD0N1Xw .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-AjpIYA3CwdD0N1Xw .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-AjpIYA3CwdD0N1Xw .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-AjpIYA3CwdD0N1Xw .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-AjpIYA3CwdD0N1Xw .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-AjpIYA3CwdD0N1Xw .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-AjpIYA3CwdD0N1Xw .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-AjpIYA3CwdD0N1Xw .marker{fill:#333333;stroke:#333333;}#mermaid-svg-AjpIYA3CwdD0N1Xw .marker.cross{stroke:#333333;}#mermaid-svg-AjpIYA3CwdD0N1Xw svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-AjpIYA3CwdD0N1Xw p{margin:0;}#mermaid-svg-AjpIYA3CwdD0N1Xw .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-AjpIYA3CwdD0N1Xw .cluster-label text{fill:#333;}#mermaid-svg-AjpIYA3CwdD0N1Xw .cluster-label span{color:#333;}#mermaid-svg-AjpIYA3CwdD0N1Xw .cluster-label span p{background-color:transparent;}#mermaid-svg-AjpIYA3CwdD0N1Xw .label text,#mermaid-svg-AjpIYA3CwdD0N1Xw span{fill:#333;color:#333;}#mermaid-svg-AjpIYA3CwdD0N1Xw .node rect,#mermaid-svg-AjpIYA3CwdD0N1Xw .node circle,#mermaid-svg-AjpIYA3CwdD0N1Xw .node ellipse,#mermaid-svg-AjpIYA3CwdD0N1Xw .node polygon,#mermaid-svg-AjpIYA3CwdD0N1Xw .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-AjpIYA3CwdD0N1Xw .rough-node .label text,#mermaid-svg-AjpIYA3CwdD0N1Xw .node .label text,#mermaid-svg-AjpIYA3CwdD0N1Xw .image-shape .label,#mermaid-svg-AjpIYA3CwdD0N1Xw .icon-shape .label{text-anchor:middle;}#mermaid-svg-AjpIYA3CwdD0N1Xw .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-AjpIYA3CwdD0N1Xw .rough-node .label,#mermaid-svg-AjpIYA3CwdD0N1Xw .node .label,#mermaid-svg-AjpIYA3CwdD0N1Xw .image-shape .label,#mermaid-svg-AjpIYA3CwdD0N1Xw .icon-shape .label{text-align:center;}#mermaid-svg-AjpIYA3CwdD0N1Xw .node.clickable{cursor:pointer;}#mermaid-svg-AjpIYA3CwdD0N1Xw .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-AjpIYA3CwdD0N1Xw .arrowheadPath{fill:#333333;}#mermaid-svg-AjpIYA3CwdD0N1Xw .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-AjpIYA3CwdD0N1Xw .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-AjpIYA3CwdD0N1Xw .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-AjpIYA3CwdD0N1Xw .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-AjpIYA3CwdD0N1Xw .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-AjpIYA3CwdD0N1Xw .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-AjpIYA3CwdD0N1Xw .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-AjpIYA3CwdD0N1Xw .cluster text{fill:#333;}#mermaid-svg-AjpIYA3CwdD0N1Xw .cluster span{color:#333;}#mermaid-svg-AjpIYA3CwdD0N1Xw div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-AjpIYA3CwdD0N1Xw .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-AjpIYA3CwdD0N1Xw rect.text{fill:none;stroke-width:0;}#mermaid-svg-AjpIYA3CwdD0N1Xw .icon-shape,#mermaid-svg-AjpIYA3CwdD0N1Xw .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-AjpIYA3CwdD0N1Xw .icon-shape p,#mermaid-svg-AjpIYA3CwdD0N1Xw .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-AjpIYA3CwdD0N1Xw .icon-shape .label rect,#mermaid-svg-AjpIYA3CwdD0N1Xw .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-AjpIYA3CwdD0N1Xw .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-AjpIYA3CwdD0N1Xw .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-AjpIYA3CwdD0N1Xw :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 业务服务集群
Filebeat
Kafka 日志主题
Logstash 消费
Elasticsearch 集群
Kibana
生命周期策略
这套方案里，每个组件的职责要清楚：
| 环节 | 重点 |
|---|---|
| 应用 | 打结构化日志，带 traceId |
| Filebeat | 轻量采集，保证日志送出去 |
| Kafka | 削峰和缓冲 |
| Logstash | 解析、清洗、补字段 |
| Elasticsearch | 索引、存储、检索 |
| Kibana | 搜索、展示、告警辅助 |
常见坑
日志采集平台最常见的坑，不在于组件不会装，而在于规范没定好。
| 坑 | 后果 | 建议 |
|---|---|---|
| 日志格式不统一 | 字段无法检索 | 统一 JSON 格式 |
| 没有 traceId | 跨服务无法串联 | 网关入口生成并透传 |
| ERROR 滥用 | 告警噪声很大 | 明确日志级别规范 |
| 打印敏感信息 | 安全风险 | 日志输出前脱敏 |
| 日志量无限增长 | 存储成本失控 | 设置保留周期 |
| 多行异常未合并 | 堆栈被拆散 | 配置多行规则 |
| 只采集不治理 | 平台越用越乱 | 定期清理索引和字段 |
面试或方案设计怎么讲
如果被问到"日志采集怎么做"，不要只回答"用 ELK"。
更完整的回答可以这样组织：
#mermaid-svg-OIMslupz5s0RWVUa{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-OIMslupz5s0RWVUa .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-OIMslupz5s0RWVUa .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-OIMslupz5s0RWVUa .error-icon{fill:#552222;}#mermaid-svg-OIMslupz5s0RWVUa .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-OIMslupz5s0RWVUa .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-OIMslupz5s0RWVUa .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-OIMslupz5s0RWVUa .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-OIMslupz5s0RWVUa .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-OIMslupz5s0RWVUa .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-OIMslupz5s0RWVUa .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-OIMslupz5s0RWVUa .marker{fill:#333333;stroke:#333333;}#mermaid-svg-OIMslupz5s0RWVUa .marker.cross{stroke:#333333;}#mermaid-svg-OIMslupz5s0RWVUa svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-OIMslupz5s0RWVUa p{margin:0;}#mermaid-svg-OIMslupz5s0RWVUa .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-OIMslupz5s0RWVUa .cluster-label text{fill:#333;}#mermaid-svg-OIMslupz5s0RWVUa .cluster-label span{color:#333;}#mermaid-svg-OIMslupz5s0RWVUa .cluster-label span p{background-color:transparent;}#mermaid-svg-OIMslupz5s0RWVUa .label text,#mermaid-svg-OIMslupz5s0RWVUa span{fill:#333;color:#333;}#mermaid-svg-OIMslupz5s0RWVUa .node rect,#mermaid-svg-OIMslupz5s0RWVUa .node circle,#mermaid-svg-OIMslupz5s0RWVUa .node ellipse,#mermaid-svg-OIMslupz5s0RWVUa .node polygon,#mermaid-svg-OIMslupz5s0RWVUa .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-OIMslupz5s0RWVUa .rough-node .label text,#mermaid-svg-OIMslupz5s0RWVUa .node .label text,#mermaid-svg-OIMslupz5s0RWVUa .image-shape .label,#mermaid-svg-OIMslupz5s0RWVUa .icon-shape .label{text-anchor:middle;}#mermaid-svg-OIMslupz5s0RWVUa .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-OIMslupz5s0RWVUa .rough-node .label,#mermaid-svg-OIMslupz5s0RWVUa .node .label,#mermaid-svg-OIMslupz5s0RWVUa .image-shape .label,#mermaid-svg-OIMslupz5s0RWVUa .icon-shape .label{text-align:center;}#mermaid-svg-OIMslupz5s0RWVUa .node.clickable{cursor:pointer;}#mermaid-svg-OIMslupz5s0RWVUa .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-OIMslupz5s0RWVUa .arrowheadPath{fill:#333333;}#mermaid-svg-OIMslupz5s0RWVUa .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-OIMslupz5s0RWVUa .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-OIMslupz5s0RWVUa .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-OIMslupz5s0RWVUa .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-OIMslupz5s0RWVUa .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-OIMslupz5s0RWVUa .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-OIMslupz5s0RWVUa .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-OIMslupz5s0RWVUa .cluster text{fill:#333;}#mermaid-svg-OIMslupz5s0RWVUa .cluster span{color:#333;}#mermaid-svg-OIMslupz5s0RWVUa div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-OIMslupz5s0RWVUa .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-OIMslupz5s0RWVUa rect.text{fill:none;stroke-width:0;}#mermaid-svg-OIMslupz5s0RWVUa .icon-shape,#mermaid-svg-OIMslupz5s0RWVUa .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-OIMslupz5s0RWVUa .icon-shape p,#mermaid-svg-OIMslupz5s0RWVUa .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-OIMslupz5s0RWVUa .icon-shape .label rect,#mermaid-svg-OIMslupz5s0RWVUa .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-OIMslupz5s0RWVUa .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-OIMslupz5s0RWVUa .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-OIMslupz5s0RWVUa :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 说明问题背景
日志分散和排查困难
说明整体架构
应用输出结构化日志
Filebeat 采集
Logstash 解析
Elasticsearch 存储
Kibana 检索
补充 traceId 和脱敏
说明高并发下引入 Kafka
可以按这段话讲：
我们会先规范应用日志格式，日志里必须包含时间、级别、服务名、traceId、业务主键和异常信息。应用通过 Logback 输出到本地文件，Filebeat 负责采集日志。如果日志需要解析和清洗，就发送到 Logstash，再写入 Elasticsearch，最后通过 Kibana 查询。日志量比较大时，中间会接 Kafka 做缓冲，避免日志高峰直接打爆后端。安全上会做敏感字段脱敏，存储上会按服务、环境、日期建索引并设置保留周期。
这比只说组件名要强很多，因为它说清楚了数据怎么流、每个组件干什么、生产上有哪些风险点。
最后
日志系统不是为了"看起来很完整"，而是为了让问题发生时能更快定位。
一套好用的日志平台，应该做到三件事：
- 一次请求能靠 traceId 串起来。
- 一个异常能靠服务名、时间、业务主键快速定位。
- 日志链路本身要稳定、安全、可控。
做到这三点，日志就不只是文件，而是生产系统的黑匣子。