生产问题排查与性能瓶颈定位：日志、监控、链路追踪、压测与Arthas

生产问题最怕的不是报错，而是没人知道从哪下手。

一个接口突然变慢，用户说提交订单失败，监控开始报警，日志里一堆异常，CPU 又有点高。这个时候如果只是凭感觉去猜，很容易越查越乱。

生产排障要有顺序：先确认影响范围，再拿日志、监控、链路追踪定位层级，然后用压测或复现缩小问题，必要时用 Arthas 进入 JVM 内部观察，最后修复、验证、复盘。

先把问题说清楚

遇到生产问题，第一步不是打开代码，而是把问题描述清楚。

至少要回答这几个问题：

问题	目的
什么时间开始	缩小日志和监控时间范围
影响哪些用户	判断是否全量故障
哪些接口异常	定位业务入口
表现是什么	区分慢、错、卡、丢数据
是否刚发布	判断和变更是否相关
是否有外部依赖	判断第三方、数据库、缓存、消息队列是否异常

可以用这个流程先收口：
#mermaid-svg-zluAKJsQyIIFHT3y{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-zluAKJsQyIIFHT3y .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-zluAKJsQyIIFHT3y .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-zluAKJsQyIIFHT3y .error-icon{fill:#552222;}#mermaid-svg-zluAKJsQyIIFHT3y .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-zluAKJsQyIIFHT3y .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-zluAKJsQyIIFHT3y .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-zluAKJsQyIIFHT3y .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-zluAKJsQyIIFHT3y .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-zluAKJsQyIIFHT3y .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-zluAKJsQyIIFHT3y .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-zluAKJsQyIIFHT3y .marker{fill:#333333;stroke:#333333;}#mermaid-svg-zluAKJsQyIIFHT3y .marker.cross{stroke:#333333;}#mermaid-svg-zluAKJsQyIIFHT3y svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-zluAKJsQyIIFHT3y p{margin:0;}#mermaid-svg-zluAKJsQyIIFHT3y .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-zluAKJsQyIIFHT3y .cluster-label text{fill:#333;}#mermaid-svg-zluAKJsQyIIFHT3y .cluster-label span{color:#333;}#mermaid-svg-zluAKJsQyIIFHT3y .cluster-label span p{background-color:transparent;}#mermaid-svg-zluAKJsQyIIFHT3y .label text,#mermaid-svg-zluAKJsQyIIFHT3y span{fill:#333;color:#333;}#mermaid-svg-zluAKJsQyIIFHT3y .node rect,#mermaid-svg-zluAKJsQyIIFHT3y .node circle,#mermaid-svg-zluAKJsQyIIFHT3y .node ellipse,#mermaid-svg-zluAKJsQyIIFHT3y .node polygon,#mermaid-svg-zluAKJsQyIIFHT3y .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-zluAKJsQyIIFHT3y .rough-node .label text,#mermaid-svg-zluAKJsQyIIFHT3y .node .label text,#mermaid-svg-zluAKJsQyIIFHT3y .image-shape .label,#mermaid-svg-zluAKJsQyIIFHT3y .icon-shape .label{text-anchor:middle;}#mermaid-svg-zluAKJsQyIIFHT3y .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-zluAKJsQyIIFHT3y .rough-node .label,#mermaid-svg-zluAKJsQyIIFHT3y .node .label,#mermaid-svg-zluAKJsQyIIFHT3y .image-shape .label,#mermaid-svg-zluAKJsQyIIFHT3y .icon-shape .label{text-align:center;}#mermaid-svg-zluAKJsQyIIFHT3y .node.clickable{cursor:pointer;}#mermaid-svg-zluAKJsQyIIFHT3y .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-zluAKJsQyIIFHT3y .arrowheadPath{fill:#333333;}#mermaid-svg-zluAKJsQyIIFHT3y .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-zluAKJsQyIIFHT3y .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-zluAKJsQyIIFHT3y .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-zluAKJsQyIIFHT3y .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-zluAKJsQyIIFHT3y .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-zluAKJsQyIIFHT3y .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-zluAKJsQyIIFHT3y .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-zluAKJsQyIIFHT3y .cluster text{fill:#333;}#mermaid-svg-zluAKJsQyIIFHT3y .cluster span{color:#333;}#mermaid-svg-zluAKJsQyIIFHT3y div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-zluAKJsQyIIFHT3y .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-zluAKJsQyIIFHT3y rect.text{fill:none;stroke-width:0;}#mermaid-svg-zluAKJsQyIIFHT3y .icon-shape,#mermaid-svg-zluAKJsQyIIFHT3y .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-zluAKJsQyIIFHT3y .icon-shape p,#mermaid-svg-zluAKJsQyIIFHT3y .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-zluAKJsQyIIFHT3y .icon-shape .label rect,#mermaid-svg-zluAKJsQyIIFHT3y .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-zluAKJsQyIIFHT3y .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-zluAKJsQyIIFHT3y .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-zluAKJsQyIIFHT3y :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 收到告警或用户反馈
确认异常时间
确认影响范围
确认异常接口或业务动作
查看近期变更
进入日志、监控、链路追踪分析

不要一上来就问"是不是数据库慢"。数据库当然可能慢，但也可能是线程池打满、缓存击穿、远程服务超时、锁竞争、GC 停顿、发布引入了死循环。

排障最重要的是用证据缩小范围。

一条通用排障主线

生产问题可以按"现象、范围、层级、原因、修复、验证"来推进。
#mermaid-svg-IKJWpuuPsegucgZl{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-IKJWpuuPsegucgZl .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-IKJWpuuPsegucgZl .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-IKJWpuuPsegucgZl .error-icon{fill:#552222;}#mermaid-svg-IKJWpuuPsegucgZl .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-IKJWpuuPsegucgZl .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-IKJWpuuPsegucgZl .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-IKJWpuuPsegucgZl .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-IKJWpuuPsegucgZl .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-IKJWpuuPsegucgZl .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-IKJWpuuPsegucgZl .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-IKJWpuuPsegucgZl .marker{fill:#333333;stroke:#333333;}#mermaid-svg-IKJWpuuPsegucgZl .marker.cross{stroke:#333333;}#mermaid-svg-IKJWpuuPsegucgZl svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-IKJWpuuPsegucgZl p{margin:0;}#mermaid-svg-IKJWpuuPsegucgZl .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-IKJWpuuPsegucgZl .cluster-label text{fill:#333;}#mermaid-svg-IKJWpuuPsegucgZl .cluster-label span{color:#333;}#mermaid-svg-IKJWpuuPsegucgZl .cluster-label span p{background-color:transparent;}#mermaid-svg-IKJWpuuPsegucgZl .label text,#mermaid-svg-IKJWpuuPsegucgZl span{fill:#333;color:#333;}#mermaid-svg-IKJWpuuPsegucgZl .node rect,#mermaid-svg-IKJWpuuPsegucgZl .node circle,#mermaid-svg-IKJWpuuPsegucgZl .node ellipse,#mermaid-svg-IKJWpuuPsegucgZl .node polygon,#mermaid-svg-IKJWpuuPsegucgZl .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-IKJWpuuPsegucgZl .rough-node .label text,#mermaid-svg-IKJWpuuPsegucgZl .node .label text,#mermaid-svg-IKJWpuuPsegucgZl .image-shape .label,#mermaid-svg-IKJWpuuPsegucgZl .icon-shape .label{text-anchor:middle;}#mermaid-svg-IKJWpuuPsegucgZl .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-IKJWpuuPsegucgZl .rough-node .label,#mermaid-svg-IKJWpuuPsegucgZl .node .label,#mermaid-svg-IKJWpuuPsegucgZl .image-shape .label,#mermaid-svg-IKJWpuuPsegucgZl .icon-shape .label{text-align:center;}#mermaid-svg-IKJWpuuPsegucgZl .node.clickable{cursor:pointer;}#mermaid-svg-IKJWpuuPsegucgZl .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-IKJWpuuPsegucgZl .arrowheadPath{fill:#333333;}#mermaid-svg-IKJWpuuPsegucgZl .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-IKJWpuuPsegucgZl .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-IKJWpuuPsegucgZl .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-IKJWpuuPsegucgZl .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-IKJWpuuPsegucgZl .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-IKJWpuuPsegucgZl .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-IKJWpuuPsegucgZl .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-IKJWpuuPsegucgZl .cluster text{fill:#333;}#mermaid-svg-IKJWpuuPsegucgZl .cluster span{color:#333;}#mermaid-svg-IKJWpuuPsegucgZl div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-IKJWpuuPsegucgZl .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-IKJWpuuPsegucgZl rect.text{fill:none;stroke-width:0;}#mermaid-svg-IKJWpuuPsegucgZl .icon-shape,#mermaid-svg-IKJWpuuPsegucgZl .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-IKJWpuuPsegucgZl .icon-shape p,#mermaid-svg-IKJWpuuPsegucgZl .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-IKJWpuuPsegucgZl .icon-shape .label rect,#mermaid-svg-IKJWpuuPsegucgZl .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-IKJWpuuPsegucgZl .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-IKJWpuuPsegucgZl .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-IKJWpuuPsegucgZl :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 现象
慢、错、超时、资源高
范围
单接口、单服务、单机器、全链路
层级
应用、JVM、数据库、缓存、消息队列、网络
原因
配置、代码、流量、数据、依赖、资源
修复
回滚、降级、扩容、限流、修代码
验证
指标恢复、错误下降、用户路径正常

这个顺序很笨，但很稳。它避免了一个常见问题：还没确认影响范围，就开始改代码；还没确认瓶颈层级，就开始加机器。

日志看什么

日志适合回答"发生了什么"。

排查生产问题时，先用时间范围和 traceId 收敛日志。
#mermaid-svg-8zSjxHXBtgZJnvXY{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-8zSjxHXBtgZJnvXY .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-8zSjxHXBtgZJnvXY .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-8zSjxHXBtgZJnvXY .error-icon{fill:#552222;}#mermaid-svg-8zSjxHXBtgZJnvXY .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-8zSjxHXBtgZJnvXY .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-8zSjxHXBtgZJnvXY .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-8zSjxHXBtgZJnvXY .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-8zSjxHXBtgZJnvXY .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-8zSjxHXBtgZJnvXY .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-8zSjxHXBtgZJnvXY .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-8zSjxHXBtgZJnvXY .marker{fill:#333333;stroke:#333333;}#mermaid-svg-8zSjxHXBtgZJnvXY .marker.cross{stroke:#333333;}#mermaid-svg-8zSjxHXBtgZJnvXY svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-8zSjxHXBtgZJnvXY p{margin:0;}#mermaid-svg-8zSjxHXBtgZJnvXY .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-8zSjxHXBtgZJnvXY .cluster-label text{fill:#333;}#mermaid-svg-8zSjxHXBtgZJnvXY .cluster-label span{color:#333;}#mermaid-svg-8zSjxHXBtgZJnvXY .cluster-label span p{background-color:transparent;}#mermaid-svg-8zSjxHXBtgZJnvXY .label text,#mermaid-svg-8zSjxHXBtgZJnvXY span{fill:#333;color:#333;}#mermaid-svg-8zSjxHXBtgZJnvXY .node rect,#mermaid-svg-8zSjxHXBtgZJnvXY .node circle,#mermaid-svg-8zSjxHXBtgZJnvXY .node ellipse,#mermaid-svg-8zSjxHXBtgZJnvXY .node polygon,#mermaid-svg-8zSjxHXBtgZJnvXY .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-8zSjxHXBtgZJnvXY .rough-node .label text,#mermaid-svg-8zSjxHXBtgZJnvXY .node .label text,#mermaid-svg-8zSjxHXBtgZJnvXY .image-shape .label,#mermaid-svg-8zSjxHXBtgZJnvXY .icon-shape .label{text-anchor:middle;}#mermaid-svg-8zSjxHXBtgZJnvXY .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-8zSjxHXBtgZJnvXY .rough-node .label,#mermaid-svg-8zSjxHXBtgZJnvXY .node .label,#mermaid-svg-8zSjxHXBtgZJnvXY .image-shape .label,#mermaid-svg-8zSjxHXBtgZJnvXY .icon-shape .label{text-align:center;}#mermaid-svg-8zSjxHXBtgZJnvXY .node.clickable{cursor:pointer;}#mermaid-svg-8zSjxHXBtgZJnvXY .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-8zSjxHXBtgZJnvXY .arrowheadPath{fill:#333333;}#mermaid-svg-8zSjxHXBtgZJnvXY .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-8zSjxHXBtgZJnvXY .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-8zSjxHXBtgZJnvXY .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-8zSjxHXBtgZJnvXY .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-8zSjxHXBtgZJnvXY .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-8zSjxHXBtgZJnvXY .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-8zSjxHXBtgZJnvXY .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-8zSjxHXBtgZJnvXY .cluster text{fill:#333;}#mermaid-svg-8zSjxHXBtgZJnvXY .cluster span{color:#333;}#mermaid-svg-8zSjxHXBtgZJnvXY div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-8zSjxHXBtgZJnvXY .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-8zSjxHXBtgZJnvXY rect.text{fill:none;stroke-width:0;}#mermaid-svg-8zSjxHXBtgZJnvXY .icon-shape,#mermaid-svg-8zSjxHXBtgZJnvXY .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-8zSjxHXBtgZJnvXY .icon-shape p,#mermaid-svg-8zSjxHXBtgZJnvXY .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-8zSjxHXBtgZJnvXY .icon-shape .label rect,#mermaid-svg-8zSjxHXBtgZJnvXY .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-8zSjxHXBtgZJnvXY .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-8zSjxHXBtgZJnvXY .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-8zSjxHXBtgZJnvXY :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 异常时间
日志检索条件
traceId 或业务主键
异常栈
关键业务日志
上下游调用日志

常见查询条件包括：

条件	示例
时间	2026-06-08 10:00:00 到 10:10:00
服务名	order-service
级别	ERROR、WARN
traceId	a1b2c3d4
业务主键	orderId=10086
异常类型	TimeoutException

日志里重点看几类信息：

日志类型	关注点
异常栈	报错位置和异常类型
入口日志	请求参数、用户、接口
出口日志	调用下游的耗时和返回码
慢调用日志	哪一步耗时最高
业务状态日志	数据状态有没有异常跳变

一个好的排障日志应该能回答：

1. 请求有没有进来。
2. 请求在哪一步失败。
3. 失败前后关键业务状态是什么。
4. 下游调用是否成功，耗时多少。
5. 异常是否集中在某个服务、某台机器、某类用户。

如果日志只能看到"操作失败"，看不到业务主键和调用耗时，那不是排障能力差，而是日志本身没设计好。

监控看什么

监控适合回答"系统状态是否异常"。

排查时不要只看 CPU。CPU 高只是现象，不一定是根因。
#mermaid-svg-Z2P4tn7psXvVRyHi{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-Z2P4tn7psXvVRyHi .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-Z2P4tn7psXvVRyHi .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-Z2P4tn7psXvVRyHi .error-icon{fill:#552222;}#mermaid-svg-Z2P4tn7psXvVRyHi .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-Z2P4tn7psXvVRyHi .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-Z2P4tn7psXvVRyHi .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-Z2P4tn7psXvVRyHi .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-Z2P4tn7psXvVRyHi .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-Z2P4tn7psXvVRyHi .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-Z2P4tn7psXvVRyHi .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-Z2P4tn7psXvVRyHi .marker{fill:#333333;stroke:#333333;}#mermaid-svg-Z2P4tn7psXvVRyHi .marker.cross{stroke:#333333;}#mermaid-svg-Z2P4tn7psXvVRyHi svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-Z2P4tn7psXvVRyHi p{margin:0;}#mermaid-svg-Z2P4tn7psXvVRyHi .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-Z2P4tn7psXvVRyHi .cluster-label text{fill:#333;}#mermaid-svg-Z2P4tn7psXvVRyHi .cluster-label span{color:#333;}#mermaid-svg-Z2P4tn7psXvVRyHi .cluster-label span p{background-color:transparent;}#mermaid-svg-Z2P4tn7psXvVRyHi .label text,#mermaid-svg-Z2P4tn7psXvVRyHi span{fill:#333;color:#333;}#mermaid-svg-Z2P4tn7psXvVRyHi .node rect,#mermaid-svg-Z2P4tn7psXvVRyHi .node circle,#mermaid-svg-Z2P4tn7psXvVRyHi .node ellipse,#mermaid-svg-Z2P4tn7psXvVRyHi .node polygon,#mermaid-svg-Z2P4tn7psXvVRyHi .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-Z2P4tn7psXvVRyHi .rough-node .label text,#mermaid-svg-Z2P4tn7psXvVRyHi .node .label text,#mermaid-svg-Z2P4tn7psXvVRyHi .image-shape .label,#mermaid-svg-Z2P4tn7psXvVRyHi .icon-shape .label{text-anchor:middle;}#mermaid-svg-Z2P4tn7psXvVRyHi .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-Z2P4tn7psXvVRyHi .rough-node .label,#mermaid-svg-Z2P4tn7psXvVRyHi .node .label,#mermaid-svg-Z2P4tn7psXvVRyHi .image-shape .label,#mermaid-svg-Z2P4tn7psXvVRyHi .icon-shape .label{text-align:center;}#mermaid-svg-Z2P4tn7psXvVRyHi .node.clickable{cursor:pointer;}#mermaid-svg-Z2P4tn7psXvVRyHi .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-Z2P4tn7psXvVRyHi .arrowheadPath{fill:#333333;}#mermaid-svg-Z2P4tn7psXvVRyHi .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-Z2P4tn7psXvVRyHi .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-Z2P4tn7psXvVRyHi .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-Z2P4tn7psXvVRyHi .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-Z2P4tn7psXvVRyHi .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-Z2P4tn7psXvVRyHi .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-Z2P4tn7psXvVRyHi .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-Z2P4tn7psXvVRyHi .cluster text{fill:#333;}#mermaid-svg-Z2P4tn7psXvVRyHi .cluster span{color:#333;}#mermaid-svg-Z2P4tn7psXvVRyHi div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-Z2P4tn7psXvVRyHi .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-Z2P4tn7psXvVRyHi rect.text{fill:none;stroke-width:0;}#mermaid-svg-Z2P4tn7psXvVRyHi .icon-shape,#mermaid-svg-Z2P4tn7psXvVRyHi .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-Z2P4tn7psXvVRyHi .icon-shape p,#mermaid-svg-Z2P4tn7psXvVRyHi .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-Z2P4tn7psXvVRyHi .icon-shape .label rect,#mermaid-svg-Z2P4tn7psXvVRyHi .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-Z2P4tn7psXvVRyHi .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-Z2P4tn7psXvVRyHi .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-Z2P4tn7psXvVRyHi :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 监控指标
系统资源
JVM
应用接口
中间件
CPU、内存、磁盘 IO、网络
堆内存、GC、线程数
QPS、响应时间、错误率
数据库、缓存、消息队列

常见指标可以这样看：

指标	可能问题
CPU 高	计算密集、死循环、频繁 GC、线程竞争
内存上涨	缓存过大、对象堆积、内存泄漏
Full GC 频繁	堆空间不足、对象生命周期异常
线程数过高	线程池配置不合理、阻塞调用多
磁盘 IO 高	日志过量、慢查询、大文件读写
网络延迟高	下游依赖慢、跨机房访问、网络拥塞
错误率升高	业务异常、依赖失败、发布缺陷
响应时间升高	数据库慢、锁竞争、远程调用慢

监控的核心不是看一个点，而是看趋势和关联。

例如接口变慢时，应该同时看：
#mermaid-svg-U9lAxwMsX4GHoIcK{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-U9lAxwMsX4GHoIcK .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-U9lAxwMsX4GHoIcK .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-U9lAxwMsX4GHoIcK .error-icon{fill:#552222;}#mermaid-svg-U9lAxwMsX4GHoIcK .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-U9lAxwMsX4GHoIcK .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-U9lAxwMsX4GHoIcK .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-U9lAxwMsX4GHoIcK .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-U9lAxwMsX4GHoIcK .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-U9lAxwMsX4GHoIcK .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-U9lAxwMsX4GHoIcK .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-U9lAxwMsX4GHoIcK .marker{fill:#333333;stroke:#333333;}#mermaid-svg-U9lAxwMsX4GHoIcK .marker.cross{stroke:#333333;}#mermaid-svg-U9lAxwMsX4GHoIcK svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-U9lAxwMsX4GHoIcK p{margin:0;}#mermaid-svg-U9lAxwMsX4GHoIcK .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-U9lAxwMsX4GHoIcK .cluster-label text{fill:#333;}#mermaid-svg-U9lAxwMsX4GHoIcK .cluster-label span{color:#333;}#mermaid-svg-U9lAxwMsX4GHoIcK .cluster-label span p{background-color:transparent;}#mermaid-svg-U9lAxwMsX4GHoIcK .label text,#mermaid-svg-U9lAxwMsX4GHoIcK span{fill:#333;color:#333;}#mermaid-svg-U9lAxwMsX4GHoIcK .node rect,#mermaid-svg-U9lAxwMsX4GHoIcK .node circle,#mermaid-svg-U9lAxwMsX4GHoIcK .node ellipse,#mermaid-svg-U9lAxwMsX4GHoIcK .node polygon,#mermaid-svg-U9lAxwMsX4GHoIcK .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-U9lAxwMsX4GHoIcK .rough-node .label text,#mermaid-svg-U9lAxwMsX4GHoIcK .node .label text,#mermaid-svg-U9lAxwMsX4GHoIcK .image-shape .label,#mermaid-svg-U9lAxwMsX4GHoIcK .icon-shape .label{text-anchor:middle;}#mermaid-svg-U9lAxwMsX4GHoIcK .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-U9lAxwMsX4GHoIcK .rough-node .label,#mermaid-svg-U9lAxwMsX4GHoIcK .node .label,#mermaid-svg-U9lAxwMsX4GHoIcK .image-shape .label,#mermaid-svg-U9lAxwMsX4GHoIcK .icon-shape .label{text-align:center;}#mermaid-svg-U9lAxwMsX4GHoIcK .node.clickable{cursor:pointer;}#mermaid-svg-U9lAxwMsX4GHoIcK .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-U9lAxwMsX4GHoIcK .arrowheadPath{fill:#333333;}#mermaid-svg-U9lAxwMsX4GHoIcK .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-U9lAxwMsX4GHoIcK .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-U9lAxwMsX4GHoIcK .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-U9lAxwMsX4GHoIcK .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-U9lAxwMsX4GHoIcK .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-U9lAxwMsX4GHoIcK .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-U9lAxwMsX4GHoIcK .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-U9lAxwMsX4GHoIcK .cluster text{fill:#333;}#mermaid-svg-U9lAxwMsX4GHoIcK .cluster span{color:#333;}#mermaid-svg-U9lAxwMsX4GHoIcK div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-U9lAxwMsX4GHoIcK .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-U9lAxwMsX4GHoIcK rect.text{fill:none;stroke-width:0;}#mermaid-svg-U9lAxwMsX4GHoIcK .icon-shape,#mermaid-svg-U9lAxwMsX4GHoIcK .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-U9lAxwMsX4GHoIcK .icon-shape p,#mermaid-svg-U9lAxwMsX4GHoIcK .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-U9lAxwMsX4GHoIcK .icon-shape .label rect,#mermaid-svg-U9lAxwMsX4GHoIcK .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-U9lAxwMsX4GHoIcK .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-U9lAxwMsX4GHoIcK .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-U9lAxwMsX4GHoIcK :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 接口响应时间升高
QPS 是否升高
错误率是否升高
CPU 是否升高
GC 是否异常
数据库连接池是否打满
下游接口是否变慢

如果 QPS 没变，CPU 没变，但数据库慢查询突然增加，很可能瓶颈在数据库。

如果 QPS 暴涨，线程池队列堆积，下游超时增多，问题可能是流量超过了服务承载能力。

如果响应时间呈周期性尖刺，同时 Full GC 频繁，就要重点看 JVM 内存。

链路追踪看什么

链路追踪适合回答"慢在哪里"。

一次请求经过多个服务时，单看日志很容易碎。链路追踪能把调用关系画出来，并显示每一段耗时。
"支付服务" "优惠券服务" "库存服务" "订单服务" "网关" "支付服务" "优惠券服务" "库存服务" "订单服务" "网关" #mermaid-svg-RfEJdOaFfd6pVx7H{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-RfEJdOaFfd6pVx7H .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-RfEJdOaFfd6pVx7H .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-RfEJdOaFfd6pVx7H .error-icon{fill:#552222;}#mermaid-svg-RfEJdOaFfd6pVx7H .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-RfEJdOaFfd6pVx7H .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-RfEJdOaFfd6pVx7H .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-RfEJdOaFfd6pVx7H .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-RfEJdOaFfd6pVx7H .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-RfEJdOaFfd6pVx7H .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-RfEJdOaFfd6pVx7H .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-RfEJdOaFfd6pVx7H .marker{fill:#333333;stroke:#333333;}#mermaid-svg-RfEJdOaFfd6pVx7H .marker.cross{stroke:#333333;}#mermaid-svg-RfEJdOaFfd6pVx7H svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-RfEJdOaFfd6pVx7H p{margin:0;}#mermaid-svg-RfEJdOaFfd6pVx7H .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-RfEJdOaFfd6pVx7H text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-RfEJdOaFfd6pVx7H .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-RfEJdOaFfd6pVx7H .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-RfEJdOaFfd6pVx7H .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-RfEJdOaFfd6pVx7H .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-RfEJdOaFfd6pVx7H #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-RfEJdOaFfd6pVx7H .sequenceNumber{fill:white;}#mermaid-svg-RfEJdOaFfd6pVx7H #sequencenumber{fill:#333;}#mermaid-svg-RfEJdOaFfd6pVx7H #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-RfEJdOaFfd6pVx7H .messageText{fill:#333;stroke:none;}#mermaid-svg-RfEJdOaFfd6pVx7H .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-RfEJdOaFfd6pVx7H .labelText,#mermaid-svg-RfEJdOaFfd6pVx7H .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-RfEJdOaFfd6pVx7H .loopText,#mermaid-svg-RfEJdOaFfd6pVx7H .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-RfEJdOaFfd6pVx7H .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-RfEJdOaFfd6pVx7H .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-RfEJdOaFfd6pVx7H .noteText,#mermaid-svg-RfEJdOaFfd6pVx7H .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-RfEJdOaFfd6pVx7H .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-RfEJdOaFfd6pVx7H .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-RfEJdOaFfd6pVx7H .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-RfEJdOaFfd6pVx7H .actorPopupMenu{position:absolute;}#mermaid-svg-RfEJdOaFfd6pVx7H .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-RfEJdOaFfd6pVx7H .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-RfEJdOaFfd6pVx7H .actor-man circle,#mermaid-svg-RfEJdOaFfd6pVx7H line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-RfEJdOaFfd6pVx7H :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} "下单请求 30ms""扣减库存 80ms""锁定优惠券 600ms""创建支付单 70ms"

看到这条链路，就不用猜了。主要耗时在优惠券服务。

链路追踪通常关注：

维度	说明
调用拓扑	请求经过哪些服务
单段耗时	哪个服务或方法最慢
错误节点	哪个下游返回错误
调用次数	是否出现重复调用
traceId	和日志关联定位细节

常见工具有 SkyWalking、Zipkin、Jaeger、OpenTelemetry 体系。工具名字不重要，重要的是系统要能把日志、指标、调用链用同一个 traceId 关联起来。
#mermaid-svg-yThaQ6m9SlQ2Ysm1{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .error-icon{fill:#552222;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .marker{fill:#333333;stroke:#333333;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .marker.cross{stroke:#333333;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 p{margin:0;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .cluster-label text{fill:#333;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .cluster-label span{color:#333;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .cluster-label span p{background-color:transparent;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .label text,#mermaid-svg-yThaQ6m9SlQ2Ysm1 span{fill:#333;color:#333;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .node rect,#mermaid-svg-yThaQ6m9SlQ2Ysm1 .node circle,#mermaid-svg-yThaQ6m9SlQ2Ysm1 .node ellipse,#mermaid-svg-yThaQ6m9SlQ2Ysm1 .node polygon,#mermaid-svg-yThaQ6m9SlQ2Ysm1 .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .rough-node .label text,#mermaid-svg-yThaQ6m9SlQ2Ysm1 .node .label text,#mermaid-svg-yThaQ6m9SlQ2Ysm1 .image-shape .label,#mermaid-svg-yThaQ6m9SlQ2Ysm1 .icon-shape .label{text-anchor:middle;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .rough-node .label,#mermaid-svg-yThaQ6m9SlQ2Ysm1 .node .label,#mermaid-svg-yThaQ6m9SlQ2Ysm1 .image-shape .label,#mermaid-svg-yThaQ6m9SlQ2Ysm1 .icon-shape .label{text-align:center;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .node.clickable{cursor:pointer;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .arrowheadPath{fill:#333333;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .cluster text{fill:#333;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .cluster span{color:#333;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 rect.text{fill:none;stroke-width:0;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .icon-shape,#mermaid-svg-yThaQ6m9SlQ2Ysm1 .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .icon-shape p,#mermaid-svg-yThaQ6m9SlQ2Ysm1 .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .icon-shape .label rect,#mermaid-svg-yThaQ6m9SlQ2Ysm1 .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-yThaQ6m9SlQ2Ysm1 :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} traceId
日志
链路追踪
指标标签
看异常细节
看调用路径和耗时
看整体趋势

有了这种关联，排查体验会完全不同。

压测用来验证瓶颈

压测不是为了跑一个漂亮数字，而是为了知道系统在哪个压力点开始撑不住。

这里要先分清压测和生产排障的边界：压测通常在上线前或压测环境做，用来验证容量和找潜在瓶颈；生产排障是在问题已经发生时，用日志、监控、链路追踪、线程栈、Arthas 等手段收集证据。不要为了定位线上问题，直接在生产环境随意压测，那很容易把故障扩大。

常见指标包括：

指标	说明
并发数	同时发起请求的用户或线程数量
QPS	每秒处理请求数
吞吐量	单位时间处理的请求或数据量
响应时间	从发出请求到收到完整响应的耗时
延迟	从发出请求到收到首个响应的耗时
错误率	失败请求占比
CPU	服务端计算资源使用情况
内存	堆内存、缓存、对象堆积情况
磁盘 IO	日志、数据库、文件读写压力
网络 IO	上下游传输压力

这里有个容易混的点：响应时间和延迟不是完全一回事。

在 JMeter 里，elapsed time 更接近一次请求从发送前到完整响应接收后的耗时；latency 更接近从发送前到收到响应第一部分的时间。实际分析时，响应时间更适合看用户整体等待，延迟更适合看服务开始响应的速度。

压测流程可以这样做：
#mermaid-svg-JmfYvhtMnGYQAtmJ{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-JmfYvhtMnGYQAtmJ .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-JmfYvhtMnGYQAtmJ .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-JmfYvhtMnGYQAtmJ .error-icon{fill:#552222;}#mermaid-svg-JmfYvhtMnGYQAtmJ .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-JmfYvhtMnGYQAtmJ .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-JmfYvhtMnGYQAtmJ .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-JmfYvhtMnGYQAtmJ .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-JmfYvhtMnGYQAtmJ .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-JmfYvhtMnGYQAtmJ .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-JmfYvhtMnGYQAtmJ .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-JmfYvhtMnGYQAtmJ .marker{fill:#333333;stroke:#333333;}#mermaid-svg-JmfYvhtMnGYQAtmJ .marker.cross{stroke:#333333;}#mermaid-svg-JmfYvhtMnGYQAtmJ svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-JmfYvhtMnGYQAtmJ p{margin:0;}#mermaid-svg-JmfYvhtMnGYQAtmJ .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-JmfYvhtMnGYQAtmJ .cluster-label text{fill:#333;}#mermaid-svg-JmfYvhtMnGYQAtmJ .cluster-label span{color:#333;}#mermaid-svg-JmfYvhtMnGYQAtmJ .cluster-label span p{background-color:transparent;}#mermaid-svg-JmfYvhtMnGYQAtmJ .label text,#mermaid-svg-JmfYvhtMnGYQAtmJ span{fill:#333;color:#333;}#mermaid-svg-JmfYvhtMnGYQAtmJ .node rect,#mermaid-svg-JmfYvhtMnGYQAtmJ .node circle,#mermaid-svg-JmfYvhtMnGYQAtmJ .node ellipse,#mermaid-svg-JmfYvhtMnGYQAtmJ .node polygon,#mermaid-svg-JmfYvhtMnGYQAtmJ .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-JmfYvhtMnGYQAtmJ .rough-node .label text,#mermaid-svg-JmfYvhtMnGYQAtmJ .node .label text,#mermaid-svg-JmfYvhtMnGYQAtmJ .image-shape .label,#mermaid-svg-JmfYvhtMnGYQAtmJ .icon-shape .label{text-anchor:middle;}#mermaid-svg-JmfYvhtMnGYQAtmJ .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-JmfYvhtMnGYQAtmJ .rough-node .label,#mermaid-svg-JmfYvhtMnGYQAtmJ .node .label,#mermaid-svg-JmfYvhtMnGYQAtmJ .image-shape .label,#mermaid-svg-JmfYvhtMnGYQAtmJ .icon-shape .label{text-align:center;}#mermaid-svg-JmfYvhtMnGYQAtmJ .node.clickable{cursor:pointer;}#mermaid-svg-JmfYvhtMnGYQAtmJ .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-JmfYvhtMnGYQAtmJ .arrowheadPath{fill:#333333;}#mermaid-svg-JmfYvhtMnGYQAtmJ .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-JmfYvhtMnGYQAtmJ .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-JmfYvhtMnGYQAtmJ .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-JmfYvhtMnGYQAtmJ .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-JmfYvhtMnGYQAtmJ .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-JmfYvhtMnGYQAtmJ .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-JmfYvhtMnGYQAtmJ .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-JmfYvhtMnGYQAtmJ .cluster text{fill:#333;}#mermaid-svg-JmfYvhtMnGYQAtmJ .cluster span{color:#333;}#mermaid-svg-JmfYvhtMnGYQAtmJ div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-JmfYvhtMnGYQAtmJ .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-JmfYvhtMnGYQAtmJ rect.text{fill:none;stroke-width:0;}#mermaid-svg-JmfYvhtMnGYQAtmJ .icon-shape,#mermaid-svg-JmfYvhtMnGYQAtmJ .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-JmfYvhtMnGYQAtmJ .icon-shape p,#mermaid-svg-JmfYvhtMnGYQAtmJ .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-JmfYvhtMnGYQAtmJ .icon-shape .label rect,#mermaid-svg-JmfYvhtMnGYQAtmJ .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-JmfYvhtMnGYQAtmJ .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-JmfYvhtMnGYQAtmJ .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-JmfYvhtMnGYQAtmJ :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 是
否
确定压测目标
准备压测环境和数据
设置并发、持续时间、阶梯增压
执行压测
观察响应时间、QPS、错误率
同时观察 CPU、内存、GC、数据库、缓存
是否出现瓶颈
定位瓶颈层级
继续加压或结束
优化后复测

压测时不要只看 JMeter 报告。客户端报告告诉你"请求表现如何"，服务端监控告诉你"为什么会这样"。

比如：

现象	可能瓶颈
QPS 上不去，CPU 很低	线程池、连接池、下游阻塞
QPS 上升后错误率升高	服务容量不足或限流触发
响应时间越来越长	队列堆积、锁竞争、数据库慢
CPU 打满	计算逻辑重、序列化开销大、死循环
Full GC 频繁	内存分配过快、对象无法回收
数据库连接池耗尽	SQL 慢、事务过长、连接未释放

压测结论要落到可执行动作上：扩容、调线程池、改 SQL、加缓存、拆热点、限流、降级、异步化，或者修代码。

Arthas 适合查什么

Arthas 是 Java 线上诊断工具，适合在不重启应用的情况下观察 JVM 内部状态。

它常用于这些场景：

场景	Arthas 能做什么
CPU 高	看线程占用和调用栈
方法慢	trace 方法耗时
参数异常	watch 方法入参和返回值
怀疑代码版本不对	jad 反编译已加载类
内存异常	看 JVM 内存和对象情况
日志级别不合适	动态查看或调整 logger

常用命令：

dashboard

查看 JVM、线程、内存、GC 等实时概览。

thread

查看 Java 线程信息。CPU 高时经常用它找热点线程。

thread -n 5

查看最忙的几个线程。

jad com.example.OrderService

反编译 JVM 里实际加载的类，确认线上运行的代码是不是你以为的版本。

trace com.example.OrderService createOrder

跟踪方法内部调用耗时，适合定位慢在哪个子调用。

watch com.example.OrderService createOrder "{params, returnObj, throwExp}" -x 3

观察方法入参、返回值和异常。

Arthas 很强，但不能乱用。trace、watch 这类命令会对目标类做字节码增强，生产环境使用时要精确指定类和方法，避免范围过大。用完要及时 reset 或 stop。
#mermaid-svg-krbV3VoDwZsd3I5X{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-krbV3VoDwZsd3I5X .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-krbV3VoDwZsd3I5X .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-krbV3VoDwZsd3I5X .error-icon{fill:#552222;}#mermaid-svg-krbV3VoDwZsd3I5X .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-krbV3VoDwZsd3I5X .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-krbV3VoDwZsd3I5X .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-krbV3VoDwZsd3I5X .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-krbV3VoDwZsd3I5X .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-krbV3VoDwZsd3I5X .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-krbV3VoDwZsd3I5X .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-krbV3VoDwZsd3I5X .marker{fill:#333333;stroke:#333333;}#mermaid-svg-krbV3VoDwZsd3I5X .marker.cross{stroke:#333333;}#mermaid-svg-krbV3VoDwZsd3I5X svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-krbV3VoDwZsd3I5X p{margin:0;}#mermaid-svg-krbV3VoDwZsd3I5X .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-krbV3VoDwZsd3I5X .cluster-label text{fill:#333;}#mermaid-svg-krbV3VoDwZsd3I5X .cluster-label span{color:#333;}#mermaid-svg-krbV3VoDwZsd3I5X .cluster-label span p{background-color:transparent;}#mermaid-svg-krbV3VoDwZsd3I5X .label text,#mermaid-svg-krbV3VoDwZsd3I5X span{fill:#333;color:#333;}#mermaid-svg-krbV3VoDwZsd3I5X .node rect,#mermaid-svg-krbV3VoDwZsd3I5X .node circle,#mermaid-svg-krbV3VoDwZsd3I5X .node ellipse,#mermaid-svg-krbV3VoDwZsd3I5X .node polygon,#mermaid-svg-krbV3VoDwZsd3I5X .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-krbV3VoDwZsd3I5X .rough-node .label text,#mermaid-svg-krbV3VoDwZsd3I5X .node .label text,#mermaid-svg-krbV3VoDwZsd3I5X .image-shape .label,#mermaid-svg-krbV3VoDwZsd3I5X .icon-shape .label{text-anchor:middle;}#mermaid-svg-krbV3VoDwZsd3I5X .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-krbV3VoDwZsd3I5X .rough-node .label,#mermaid-svg-krbV3VoDwZsd3I5X .node .label,#mermaid-svg-krbV3VoDwZsd3I5X .image-shape .label,#mermaid-svg-krbV3VoDwZsd3I5X .icon-shape .label{text-align:center;}#mermaid-svg-krbV3VoDwZsd3I5X .node.clickable{cursor:pointer;}#mermaid-svg-krbV3VoDwZsd3I5X .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-krbV3VoDwZsd3I5X .arrowheadPath{fill:#333333;}#mermaid-svg-krbV3VoDwZsd3I5X .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-krbV3VoDwZsd3I5X .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-krbV3VoDwZsd3I5X .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-krbV3VoDwZsd3I5X .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-krbV3VoDwZsd3I5X .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-krbV3VoDwZsd3I5X .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-krbV3VoDwZsd3I5X .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-krbV3VoDwZsd3I5X .cluster text{fill:#333;}#mermaid-svg-krbV3VoDwZsd3I5X .cluster span{color:#333;}#mermaid-svg-krbV3VoDwZsd3I5X div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-krbV3VoDwZsd3I5X .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-krbV3VoDwZsd3I5X rect.text{fill:none;stroke-width:0;}#mermaid-svg-krbV3VoDwZsd3I5X .icon-shape,#mermaid-svg-krbV3VoDwZsd3I5X .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-krbV3VoDwZsd3I5X .icon-shape p,#mermaid-svg-krbV3VoDwZsd3I5X .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-krbV3VoDwZsd3I5X .icon-shape .label rect,#mermaid-svg-krbV3VoDwZsd3I5X .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-krbV3VoDwZsd3I5X .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-krbV3VoDwZsd3I5X .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-krbV3VoDwZsd3I5X :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} CPU 高
方法慢
参数异常
代码版本疑问
需要 JVM 内部证据
问题类型
thread 定位热点线程
trace 查看调用耗时
watch 观察入参返回值
jad 反编译已加载类
形成证据

远程调试要谨慎

远程 debug 很好用，但它不适合随便用在生产环境。

原因很简单：断点可能让线程挂起，业务请求会被卡住；如果断在锁、事务、连接池相关位置，影响会被放大。更麻烦的是，生产数据是真实用户数据，调试过程还有权限和安全风险。

更合理的边界是：

环境	建议
本地	可以自由 debug
测试环境	可以远程 debug
预发环境	谨慎 debug，尽量模拟生产流量
生产环境	默认不使用断点式远程 debug

生产环境更推荐用日志、监控、链路追踪、Arthas、线程栈、堆转储、慢查询等方式拿证据。
#mermaid-svg-lvuKxrxkAdqzS6vJ{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-lvuKxrxkAdqzS6vJ .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-lvuKxrxkAdqzS6vJ .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-lvuKxrxkAdqzS6vJ .error-icon{fill:#552222;}#mermaid-svg-lvuKxrxkAdqzS6vJ .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-lvuKxrxkAdqzS6vJ .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-lvuKxrxkAdqzS6vJ .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-lvuKxrxkAdqzS6vJ .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-lvuKxrxkAdqzS6vJ .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-lvuKxrxkAdqzS6vJ .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-lvuKxrxkAdqzS6vJ .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-lvuKxrxkAdqzS6vJ .marker{fill:#333333;stroke:#333333;}#mermaid-svg-lvuKxrxkAdqzS6vJ .marker.cross{stroke:#333333;}#mermaid-svg-lvuKxrxkAdqzS6vJ svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-lvuKxrxkAdqzS6vJ p{margin:0;}#mermaid-svg-lvuKxrxkAdqzS6vJ .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-lvuKxrxkAdqzS6vJ .cluster-label text{fill:#333;}#mermaid-svg-lvuKxrxkAdqzS6vJ .cluster-label span{color:#333;}#mermaid-svg-lvuKxrxkAdqzS6vJ .cluster-label span p{background-color:transparent;}#mermaid-svg-lvuKxrxkAdqzS6vJ .label text,#mermaid-svg-lvuKxrxkAdqzS6vJ span{fill:#333;color:#333;}#mermaid-svg-lvuKxrxkAdqzS6vJ .node rect,#mermaid-svg-lvuKxrxkAdqzS6vJ .node circle,#mermaid-svg-lvuKxrxkAdqzS6vJ .node ellipse,#mermaid-svg-lvuKxrxkAdqzS6vJ .node polygon,#mermaid-svg-lvuKxrxkAdqzS6vJ .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-lvuKxrxkAdqzS6vJ .rough-node .label text,#mermaid-svg-lvuKxrxkAdqzS6vJ .node .label text,#mermaid-svg-lvuKxrxkAdqzS6vJ .image-shape .label,#mermaid-svg-lvuKxrxkAdqzS6vJ .icon-shape .label{text-anchor:middle;}#mermaid-svg-lvuKxrxkAdqzS6vJ .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-lvuKxrxkAdqzS6vJ .rough-node .label,#mermaid-svg-lvuKxrxkAdqzS6vJ .node .label,#mermaid-svg-lvuKxrxkAdqzS6vJ .image-shape .label,#mermaid-svg-lvuKxrxkAdqzS6vJ .icon-shape .label{text-align:center;}#mermaid-svg-lvuKxrxkAdqzS6vJ .node.clickable{cursor:pointer;}#mermaid-svg-lvuKxrxkAdqzS6vJ .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-lvuKxrxkAdqzS6vJ .arrowheadPath{fill:#333333;}#mermaid-svg-lvuKxrxkAdqzS6vJ .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-lvuKxrxkAdqzS6vJ .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-lvuKxrxkAdqzS6vJ .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-lvuKxrxkAdqzS6vJ .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-lvuKxrxkAdqzS6vJ .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-lvuKxrxkAdqzS6vJ .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-lvuKxrxkAdqzS6vJ .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-lvuKxrxkAdqzS6vJ .cluster text{fill:#333;}#mermaid-svg-lvuKxrxkAdqzS6vJ .cluster span{color:#333;}#mermaid-svg-lvuKxrxkAdqzS6vJ div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-lvuKxrxkAdqzS6vJ .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-lvuKxrxkAdqzS6vJ rect.text{fill:none;stroke-width:0;}#mermaid-svg-lvuKxrxkAdqzS6vJ .icon-shape,#mermaid-svg-lvuKxrxkAdqzS6vJ .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-lvuKxrxkAdqzS6vJ .icon-shape p,#mermaid-svg-lvuKxrxkAdqzS6vJ .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-lvuKxrxkAdqzS6vJ .icon-shape .label rect,#mermaid-svg-lvuKxrxkAdqzS6vJ .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-lvuKxrxkAdqzS6vJ .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-lvuKxrxkAdqzS6vJ .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-lvuKxrxkAdqzS6vJ :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 能
不能
生产问题
能否在非生产环境复现
测试或预发远程 debug
生产环境采集证据
日志、监控、链路追踪
Arthas、线程栈、堆转储
定位根因

如果极端情况下必须在生产环境打开诊断能力，也要满足几个条件：审批明确、范围极小、时间极短、有人值守、随时可回滚。能不用断点，就不用断点。

常见问题怎么定位

接口突然变慢

#mermaid-svg-paCSGNJf5waQqZfw{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-paCSGNJf5waQqZfw .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-paCSGNJf5waQqZfw .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-paCSGNJf5waQqZfw .error-icon{fill:#552222;}#mermaid-svg-paCSGNJf5waQqZfw .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-paCSGNJf5waQqZfw .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-paCSGNJf5waQqZfw .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-paCSGNJf5waQqZfw .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-paCSGNJf5waQqZfw .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-paCSGNJf5waQqZfw .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-paCSGNJf5waQqZfw .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-paCSGNJf5waQqZfw .marker{fill:#333333;stroke:#333333;}#mermaid-svg-paCSGNJf5waQqZfw .marker.cross{stroke:#333333;}#mermaid-svg-paCSGNJf5waQqZfw svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-paCSGNJf5waQqZfw p{margin:0;}#mermaid-svg-paCSGNJf5waQqZfw .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-paCSGNJf5waQqZfw .cluster-label text{fill:#333;}#mermaid-svg-paCSGNJf5waQqZfw .cluster-label span{color:#333;}#mermaid-svg-paCSGNJf5waQqZfw .cluster-label span p{background-color:transparent;}#mermaid-svg-paCSGNJf5waQqZfw .label text,#mermaid-svg-paCSGNJf5waQqZfw span{fill:#333;color:#333;}#mermaid-svg-paCSGNJf5waQqZfw .node rect,#mermaid-svg-paCSGNJf5waQqZfw .node circle,#mermaid-svg-paCSGNJf5waQqZfw .node ellipse,#mermaid-svg-paCSGNJf5waQqZfw .node polygon,#mermaid-svg-paCSGNJf5waQqZfw .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-paCSGNJf5waQqZfw .rough-node .label text,#mermaid-svg-paCSGNJf5waQqZfw .node .label text,#mermaid-svg-paCSGNJf5waQqZfw .image-shape .label,#mermaid-svg-paCSGNJf5waQqZfw .icon-shape .label{text-anchor:middle;}#mermaid-svg-paCSGNJf5waQqZfw .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-paCSGNJf5waQqZfw .rough-node .label,#mermaid-svg-paCSGNJf5waQqZfw .node .label,#mermaid-svg-paCSGNJf5waQqZfw .image-shape .label,#mermaid-svg-paCSGNJf5waQqZfw .icon-shape .label{text-align:center;}#mermaid-svg-paCSGNJf5waQqZfw .node.clickable{cursor:pointer;}#mermaid-svg-paCSGNJf5waQqZfw .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-paCSGNJf5waQqZfw .arrowheadPath{fill:#333333;}#mermaid-svg-paCSGNJf5waQqZfw .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-paCSGNJf5waQqZfw .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-paCSGNJf5waQqZfw .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-paCSGNJf5waQqZfw .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-paCSGNJf5waQqZfw .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-paCSGNJf5waQqZfw .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-paCSGNJf5waQqZfw .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-paCSGNJf5waQqZfw .cluster text{fill:#333;}#mermaid-svg-paCSGNJf5waQqZfw .cluster span{color:#333;}#mermaid-svg-paCSGNJf5waQqZfw div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-paCSGNJf5waQqZfw .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-paCSGNJf5waQqZfw rect.text{fill:none;stroke-width:0;}#mermaid-svg-paCSGNJf5waQqZfw .icon-shape,#mermaid-svg-paCSGNJf5waQqZfw .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-paCSGNJf5waQqZfw .icon-shape p,#mermaid-svg-paCSGNJf5waQqZfw .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-paCSGNJf5waQqZfw .icon-shape .label rect,#mermaid-svg-paCSGNJf5waQqZfw .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-paCSGNJf5waQqZfw .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-paCSGNJf5waQqZfw .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-paCSGNJf5waQqZfw :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 接口变慢
看响应时间趋势
看 QPS 是否变化
看链路追踪最慢节点
看慢节点日志
看该服务 CPU、GC、线程池、连接池
确认数据库、缓存或下游依赖

如果慢点在数据库，就查慢 SQL、索引、锁等待、连接池。

如果慢点在下游接口，就查下游错误率、超时配置、重试策略。

如果慢点在本服务，就查线程池、锁、CPU、GC、热点方法。

CPU 飙高

#mermaid-svg-IjUzVtkSMbLVrNWH{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-IjUzVtkSMbLVrNWH .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-IjUzVtkSMbLVrNWH .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-IjUzVtkSMbLVrNWH .error-icon{fill:#552222;}#mermaid-svg-IjUzVtkSMbLVrNWH .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-IjUzVtkSMbLVrNWH .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-IjUzVtkSMbLVrNWH .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-IjUzVtkSMbLVrNWH .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-IjUzVtkSMbLVrNWH .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-IjUzVtkSMbLVrNWH .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-IjUzVtkSMbLVrNWH .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-IjUzVtkSMbLVrNWH .marker{fill:#333333;stroke:#333333;}#mermaid-svg-IjUzVtkSMbLVrNWH .marker.cross{stroke:#333333;}#mermaid-svg-IjUzVtkSMbLVrNWH svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-IjUzVtkSMbLVrNWH p{margin:0;}#mermaid-svg-IjUzVtkSMbLVrNWH .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-IjUzVtkSMbLVrNWH .cluster-label text{fill:#333;}#mermaid-svg-IjUzVtkSMbLVrNWH .cluster-label span{color:#333;}#mermaid-svg-IjUzVtkSMbLVrNWH .cluster-label span p{background-color:transparent;}#mermaid-svg-IjUzVtkSMbLVrNWH .label text,#mermaid-svg-IjUzVtkSMbLVrNWH span{fill:#333;color:#333;}#mermaid-svg-IjUzVtkSMbLVrNWH .node rect,#mermaid-svg-IjUzVtkSMbLVrNWH .node circle,#mermaid-svg-IjUzVtkSMbLVrNWH .node ellipse,#mermaid-svg-IjUzVtkSMbLVrNWH .node polygon,#mermaid-svg-IjUzVtkSMbLVrNWH .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-IjUzVtkSMbLVrNWH .rough-node .label text,#mermaid-svg-IjUzVtkSMbLVrNWH .node .label text,#mermaid-svg-IjUzVtkSMbLVrNWH .image-shape .label,#mermaid-svg-IjUzVtkSMbLVrNWH .icon-shape .label{text-anchor:middle;}#mermaid-svg-IjUzVtkSMbLVrNWH .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-IjUzVtkSMbLVrNWH .rough-node .label,#mermaid-svg-IjUzVtkSMbLVrNWH .node .label,#mermaid-svg-IjUzVtkSMbLVrNWH .image-shape .label,#mermaid-svg-IjUzVtkSMbLVrNWH .icon-shape .label{text-align:center;}#mermaid-svg-IjUzVtkSMbLVrNWH .node.clickable{cursor:pointer;}#mermaid-svg-IjUzVtkSMbLVrNWH .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-IjUzVtkSMbLVrNWH .arrowheadPath{fill:#333333;}#mermaid-svg-IjUzVtkSMbLVrNWH .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-IjUzVtkSMbLVrNWH .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-IjUzVtkSMbLVrNWH .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-IjUzVtkSMbLVrNWH .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-IjUzVtkSMbLVrNWH .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-IjUzVtkSMbLVrNWH .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-IjUzVtkSMbLVrNWH .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-IjUzVtkSMbLVrNWH .cluster text{fill:#333;}#mermaid-svg-IjUzVtkSMbLVrNWH .cluster span{color:#333;}#mermaid-svg-IjUzVtkSMbLVrNWH div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-IjUzVtkSMbLVrNWH .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-IjUzVtkSMbLVrNWH rect.text{fill:none;stroke-width:0;}#mermaid-svg-IjUzVtkSMbLVrNWH .icon-shape,#mermaid-svg-IjUzVtkSMbLVrNWH .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-IjUzVtkSMbLVrNWH .icon-shape p,#mermaid-svg-IjUzVtkSMbLVrNWH .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-IjUzVtkSMbLVrNWH .icon-shape .label rect,#mermaid-svg-IjUzVtkSMbLVrNWH .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-IjUzVtkSMbLVrNWH .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-IjUzVtkSMbLVrNWH .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-IjUzVtkSMbLVrNWH :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} CPU 飙高
确认是哪台机器
查看进程 CPU
查看热点线程
转换线程 ID
查看线程栈
判断死循环、计算密集、频繁 GC 或锁竞争

可以用 Arthas 的 thread -n 5 看热点线程，也可以用系统命令配合线程栈。关键是找到具体线程正在执行的代码，而不是只说"CPU 高"。

内存持续上涨

#mermaid-svg-deaTDzYjl4wcS7pf{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-deaTDzYjl4wcS7pf .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-deaTDzYjl4wcS7pf .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-deaTDzYjl4wcS7pf .error-icon{fill:#552222;}#mermaid-svg-deaTDzYjl4wcS7pf .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-deaTDzYjl4wcS7pf .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-deaTDzYjl4wcS7pf .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-deaTDzYjl4wcS7pf .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-deaTDzYjl4wcS7pf .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-deaTDzYjl4wcS7pf .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-deaTDzYjl4wcS7pf .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-deaTDzYjl4wcS7pf .marker{fill:#333333;stroke:#333333;}#mermaid-svg-deaTDzYjl4wcS7pf .marker.cross{stroke:#333333;}#mermaid-svg-deaTDzYjl4wcS7pf svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-deaTDzYjl4wcS7pf p{margin:0;}#mermaid-svg-deaTDzYjl4wcS7pf .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-deaTDzYjl4wcS7pf .cluster-label text{fill:#333;}#mermaid-svg-deaTDzYjl4wcS7pf .cluster-label span{color:#333;}#mermaid-svg-deaTDzYjl4wcS7pf .cluster-label span p{background-color:transparent;}#mermaid-svg-deaTDzYjl4wcS7pf .label text,#mermaid-svg-deaTDzYjl4wcS7pf span{fill:#333;color:#333;}#mermaid-svg-deaTDzYjl4wcS7pf .node rect,#mermaid-svg-deaTDzYjl4wcS7pf .node circle,#mermaid-svg-deaTDzYjl4wcS7pf .node ellipse,#mermaid-svg-deaTDzYjl4wcS7pf .node polygon,#mermaid-svg-deaTDzYjl4wcS7pf .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-deaTDzYjl4wcS7pf .rough-node .label text,#mermaid-svg-deaTDzYjl4wcS7pf .node .label text,#mermaid-svg-deaTDzYjl4wcS7pf .image-shape .label,#mermaid-svg-deaTDzYjl4wcS7pf .icon-shape .label{text-anchor:middle;}#mermaid-svg-deaTDzYjl4wcS7pf .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-deaTDzYjl4wcS7pf .rough-node .label,#mermaid-svg-deaTDzYjl4wcS7pf .node .label,#mermaid-svg-deaTDzYjl4wcS7pf .image-shape .label,#mermaid-svg-deaTDzYjl4wcS7pf .icon-shape .label{text-align:center;}#mermaid-svg-deaTDzYjl4wcS7pf .node.clickable{cursor:pointer;}#mermaid-svg-deaTDzYjl4wcS7pf .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-deaTDzYjl4wcS7pf .arrowheadPath{fill:#333333;}#mermaid-svg-deaTDzYjl4wcS7pf .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-deaTDzYjl4wcS7pf .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-deaTDzYjl4wcS7pf .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-deaTDzYjl4wcS7pf .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-deaTDzYjl4wcS7pf .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-deaTDzYjl4wcS7pf .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-deaTDzYjl4wcS7pf .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-deaTDzYjl4wcS7pf .cluster text{fill:#333;}#mermaid-svg-deaTDzYjl4wcS7pf .cluster span{color:#333;}#mermaid-svg-deaTDzYjl4wcS7pf div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-deaTDzYjl4wcS7pf .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-deaTDzYjl4wcS7pf rect.text{fill:none;stroke-width:0;}#mermaid-svg-deaTDzYjl4wcS7pf .icon-shape,#mermaid-svg-deaTDzYjl4wcS7pf .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-deaTDzYjl4wcS7pf .icon-shape p,#mermaid-svg-deaTDzYjl4wcS7pf .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-deaTDzYjl4wcS7pf .icon-shape .label rect,#mermaid-svg-deaTDzYjl4wcS7pf .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-deaTDzYjl4wcS7pf .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-deaTDzYjl4wcS7pf .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-deaTDzYjl4wcS7pf :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 内存持续上涨
看堆内存曲线
看 GC 是否能回收
确认对象是否持续增长
检查缓存、集合、队列、ThreadLocal
必要时分析堆转储

内存问题要区分：

类型	表现
正常高水位	GC 后能回落
内存泄漏	GC 后仍持续上涨
瞬时大对象	某些请求触发大分配
缓存无上限	数据越跑越多

数据库慢

数据库问题不要只看一条 SQL。
#mermaid-svg-hznx7Xn7KBWl9OLg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-hznx7Xn7KBWl9OLg .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-hznx7Xn7KBWl9OLg .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-hznx7Xn7KBWl9OLg .error-icon{fill:#552222;}#mermaid-svg-hznx7Xn7KBWl9OLg .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-hznx7Xn7KBWl9OLg .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-hznx7Xn7KBWl9OLg .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-hznx7Xn7KBWl9OLg .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-hznx7Xn7KBWl9OLg .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-hznx7Xn7KBWl9OLg .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-hznx7Xn7KBWl9OLg .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-hznx7Xn7KBWl9OLg .marker{fill:#333333;stroke:#333333;}#mermaid-svg-hznx7Xn7KBWl9OLg .marker.cross{stroke:#333333;}#mermaid-svg-hznx7Xn7KBWl9OLg svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-hznx7Xn7KBWl9OLg p{margin:0;}#mermaid-svg-hznx7Xn7KBWl9OLg .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-hznx7Xn7KBWl9OLg .cluster-label text{fill:#333;}#mermaid-svg-hznx7Xn7KBWl9OLg .cluster-label span{color:#333;}#mermaid-svg-hznx7Xn7KBWl9OLg .cluster-label span p{background-color:transparent;}#mermaid-svg-hznx7Xn7KBWl9OLg .label text,#mermaid-svg-hznx7Xn7KBWl9OLg span{fill:#333;color:#333;}#mermaid-svg-hznx7Xn7KBWl9OLg .node rect,#mermaid-svg-hznx7Xn7KBWl9OLg .node circle,#mermaid-svg-hznx7Xn7KBWl9OLg .node ellipse,#mermaid-svg-hznx7Xn7KBWl9OLg .node polygon,#mermaid-svg-hznx7Xn7KBWl9OLg .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-hznx7Xn7KBWl9OLg .rough-node .label text,#mermaid-svg-hznx7Xn7KBWl9OLg .node .label text,#mermaid-svg-hznx7Xn7KBWl9OLg .image-shape .label,#mermaid-svg-hznx7Xn7KBWl9OLg .icon-shape .label{text-anchor:middle;}#mermaid-svg-hznx7Xn7KBWl9OLg .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-hznx7Xn7KBWl9OLg .rough-node .label,#mermaid-svg-hznx7Xn7KBWl9OLg .node .label,#mermaid-svg-hznx7Xn7KBWl9OLg .image-shape .label,#mermaid-svg-hznx7Xn7KBWl9OLg .icon-shape .label{text-align:center;}#mermaid-svg-hznx7Xn7KBWl9OLg .node.clickable{cursor:pointer;}#mermaid-svg-hznx7Xn7KBWl9OLg .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-hznx7Xn7KBWl9OLg .arrowheadPath{fill:#333333;}#mermaid-svg-hznx7Xn7KBWl9OLg .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-hznx7Xn7KBWl9OLg .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-hznx7Xn7KBWl9OLg .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-hznx7Xn7KBWl9OLg .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-hznx7Xn7KBWl9OLg .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-hznx7Xn7KBWl9OLg .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-hznx7Xn7KBWl9OLg .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-hznx7Xn7KBWl9OLg .cluster text{fill:#333;}#mermaid-svg-hznx7Xn7KBWl9OLg .cluster span{color:#333;}#mermaid-svg-hznx7Xn7KBWl9OLg div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-hznx7Xn7KBWl9OLg .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-hznx7Xn7KBWl9OLg rect.text{fill:none;stroke-width:0;}#mermaid-svg-hznx7Xn7KBWl9OLg .icon-shape,#mermaid-svg-hznx7Xn7KBWl9OLg .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-hznx7Xn7KBWl9OLg .icon-shape p,#mermaid-svg-hznx7Xn7KBWl9OLg .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-hznx7Xn7KBWl9OLg .icon-shape .label rect,#mermaid-svg-hznx7Xn7KBWl9OLg .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-hznx7Xn7KBWl9OLg .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-hznx7Xn7KBWl9OLg .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-hznx7Xn7KBWl9OLg :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 数据库慢
查看慢查询
检查执行计划
检查索引是否命中
检查锁等待和事务时间
检查连接池是否耗尽
结合业务流量判断

很多线上数据库问题不是 SQL 写错这么简单，而是流量上来后索引选择变差、事务持有时间过长、热点行竞争、连接池配置过小，或者应用重试把数据库打得更狠。

消息堆积

#mermaid-svg-8lcnZbKX5b76JvbF{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-8lcnZbKX5b76JvbF .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-8lcnZbKX5b76JvbF .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-8lcnZbKX5b76JvbF .error-icon{fill:#552222;}#mermaid-svg-8lcnZbKX5b76JvbF .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-8lcnZbKX5b76JvbF .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-8lcnZbKX5b76JvbF .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-8lcnZbKX5b76JvbF .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-8lcnZbKX5b76JvbF .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-8lcnZbKX5b76JvbF .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-8lcnZbKX5b76JvbF .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-8lcnZbKX5b76JvbF .marker{fill:#333333;stroke:#333333;}#mermaid-svg-8lcnZbKX5b76JvbF .marker.cross{stroke:#333333;}#mermaid-svg-8lcnZbKX5b76JvbF svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-8lcnZbKX5b76JvbF p{margin:0;}#mermaid-svg-8lcnZbKX5b76JvbF .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-8lcnZbKX5b76JvbF .cluster-label text{fill:#333;}#mermaid-svg-8lcnZbKX5b76JvbF .cluster-label span{color:#333;}#mermaid-svg-8lcnZbKX5b76JvbF .cluster-label span p{background-color:transparent;}#mermaid-svg-8lcnZbKX5b76JvbF .label text,#mermaid-svg-8lcnZbKX5b76JvbF span{fill:#333;color:#333;}#mermaid-svg-8lcnZbKX5b76JvbF .node rect,#mermaid-svg-8lcnZbKX5b76JvbF .node circle,#mermaid-svg-8lcnZbKX5b76JvbF .node ellipse,#mermaid-svg-8lcnZbKX5b76JvbF .node polygon,#mermaid-svg-8lcnZbKX5b76JvbF .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-8lcnZbKX5b76JvbF .rough-node .label text,#mermaid-svg-8lcnZbKX5b76JvbF .node .label text,#mermaid-svg-8lcnZbKX5b76JvbF .image-shape .label,#mermaid-svg-8lcnZbKX5b76JvbF .icon-shape .label{text-anchor:middle;}#mermaid-svg-8lcnZbKX5b76JvbF .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-8lcnZbKX5b76JvbF .rough-node .label,#mermaid-svg-8lcnZbKX5b76JvbF .node .label,#mermaid-svg-8lcnZbKX5b76JvbF .image-shape .label,#mermaid-svg-8lcnZbKX5b76JvbF .icon-shape .label{text-align:center;}#mermaid-svg-8lcnZbKX5b76JvbF .node.clickable{cursor:pointer;}#mermaid-svg-8lcnZbKX5b76JvbF .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-8lcnZbKX5b76JvbF .arrowheadPath{fill:#333333;}#mermaid-svg-8lcnZbKX5b76JvbF .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-8lcnZbKX5b76JvbF .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-8lcnZbKX5b76JvbF .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-8lcnZbKX5b76JvbF .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-8lcnZbKX5b76JvbF .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-8lcnZbKX5b76JvbF .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-8lcnZbKX5b76JvbF .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-8lcnZbKX5b76JvbF .cluster text{fill:#333;}#mermaid-svg-8lcnZbKX5b76JvbF .cluster span{color:#333;}#mermaid-svg-8lcnZbKX5b76JvbF div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-8lcnZbKX5b76JvbF .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-8lcnZbKX5b76JvbF rect.text{fill:none;stroke-width:0;}#mermaid-svg-8lcnZbKX5b76JvbF .icon-shape,#mermaid-svg-8lcnZbKX5b76JvbF .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-8lcnZbKX5b76JvbF .icon-shape p,#mermaid-svg-8lcnZbKX5b76JvbF .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-8lcnZbKX5b76JvbF .icon-shape .label rect,#mermaid-svg-8lcnZbKX5b76JvbF .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-8lcnZbKX5b76JvbF .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-8lcnZbKX5b76JvbF .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-8lcnZbKX5b76JvbF :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 消息堆积
确认生产速度
确认消费速度
查看消费者错误日志
检查下游依赖是否慢
判断是否扩容消费者或降级处理

消息堆积时不要只扩消费者。要先确认消费者是不是因为下游慢、数据库慢、代码异常导致消费失败。如果根因不解决，扩容只会把压力更快打到下游。

项目难点怎么表达

很多技术面试会问"项目中遇到过什么难点"。这个问题不能回答成"我用了 Redis、MQ、ELK、Arthas"，工具名不是难点。

一个好的表达结构是：
#mermaid-svg-5dLabgc3QODyMVOq{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-5dLabgc3QODyMVOq .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-5dLabgc3QODyMVOq .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-5dLabgc3QODyMVOq .error-icon{fill:#552222;}#mermaid-svg-5dLabgc3QODyMVOq .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-5dLabgc3QODyMVOq .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-5dLabgc3QODyMVOq .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-5dLabgc3QODyMVOq .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-5dLabgc3QODyMVOq .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-5dLabgc3QODyMVOq .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-5dLabgc3QODyMVOq .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-5dLabgc3QODyMVOq .marker{fill:#333333;stroke:#333333;}#mermaid-svg-5dLabgc3QODyMVOq .marker.cross{stroke:#333333;}#mermaid-svg-5dLabgc3QODyMVOq svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-5dLabgc3QODyMVOq p{margin:0;}#mermaid-svg-5dLabgc3QODyMVOq .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-5dLabgc3QODyMVOq .cluster-label text{fill:#333;}#mermaid-svg-5dLabgc3QODyMVOq .cluster-label span{color:#333;}#mermaid-svg-5dLabgc3QODyMVOq .cluster-label span p{background-color:transparent;}#mermaid-svg-5dLabgc3QODyMVOq .label text,#mermaid-svg-5dLabgc3QODyMVOq span{fill:#333;color:#333;}#mermaid-svg-5dLabgc3QODyMVOq .node rect,#mermaid-svg-5dLabgc3QODyMVOq .node circle,#mermaid-svg-5dLabgc3QODyMVOq .node ellipse,#mermaid-svg-5dLabgc3QODyMVOq .node polygon,#mermaid-svg-5dLabgc3QODyMVOq .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-5dLabgc3QODyMVOq .rough-node .label text,#mermaid-svg-5dLabgc3QODyMVOq .node .label text,#mermaid-svg-5dLabgc3QODyMVOq .image-shape .label,#mermaid-svg-5dLabgc3QODyMVOq .icon-shape .label{text-anchor:middle;}#mermaid-svg-5dLabgc3QODyMVOq .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-5dLabgc3QODyMVOq .rough-node .label,#mermaid-svg-5dLabgc3QODyMVOq .node .label,#mermaid-svg-5dLabgc3QODyMVOq .image-shape .label,#mermaid-svg-5dLabgc3QODyMVOq .icon-shape .label{text-align:center;}#mermaid-svg-5dLabgc3QODyMVOq .node.clickable{cursor:pointer;}#mermaid-svg-5dLabgc3QODyMVOq .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-5dLabgc3QODyMVOq .arrowheadPath{fill:#333333;}#mermaid-svg-5dLabgc3QODyMVOq .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-5dLabgc3QODyMVOq .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-5dLabgc3QODyMVOq .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-5dLabgc3QODyMVOq .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-5dLabgc3QODyMVOq .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-5dLabgc3QODyMVOq .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-5dLabgc3QODyMVOq .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-5dLabgc3QODyMVOq .cluster text{fill:#333;}#mermaid-svg-5dLabgc3QODyMVOq .cluster span{color:#333;}#mermaid-svg-5dLabgc3QODyMVOq div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-5dLabgc3QODyMVOq .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-5dLabgc3QODyMVOq rect.text{fill:none;stroke-width:0;}#mermaid-svg-5dLabgc3QODyMVOq .icon-shape,#mermaid-svg-5dLabgc3QODyMVOq .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-5dLabgc3QODyMVOq .icon-shape p,#mermaid-svg-5dLabgc3QODyMVOq .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-5dLabgc3QODyMVOq .icon-shape .label rect,#mermaid-svg-5dLabgc3QODyMVOq .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-5dLabgc3QODyMVOq .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-5dLabgc3QODyMVOq .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-5dLabgc3QODyMVOq :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 背景
当时系统规模和业务目标
问题
具体故障或瓶颈表现
排查过程
用了哪些证据定位
解决方案
改了什么机制
结果
指标提升或故障减少

可以按这个模板讲：

当时订单服务在活动期间出现响应时间升高，主要表现是下单接口 P95 从 200ms 涨到 1.5s，错误率也有抬升。

我先通过监控确认不是全站故障，而是订单链路变慢；再用 traceId 查日志和链路追踪，发现耗时主要集中在库存扣减和优惠券锁定；随后结合数据库慢查询和连接池监控，确认优惠券服务存在慢 SQL，并且失败重试放大了数据库压力。

处理上，我们先临时降级非核心优惠券规则，降低活动期间的下游压力；随后优化 SQL 和索引，调整连接池与超时重试策略，并补充了慢调用告警。

最终下单接口 P95 恢复到 300ms 以内，错误率回落，后续活动期间没有再出现同类故障。

这段话有几个关键点：

1. 有业务背景。
2. 有明确指标。
3. 有排查路径。
4. 有根因。
5. 有解决动作。
6. 有结果验证。

这才像真实做过项目的人讲出来的。

一套生产排障清单

最后给一个实用清单。
#mermaid-svg-SdlMHOoNcAPOvNVf{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-SdlMHOoNcAPOvNVf .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-SdlMHOoNcAPOvNVf .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-SdlMHOoNcAPOvNVf .error-icon{fill:#552222;}#mermaid-svg-SdlMHOoNcAPOvNVf .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-SdlMHOoNcAPOvNVf .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-SdlMHOoNcAPOvNVf .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-SdlMHOoNcAPOvNVf .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-SdlMHOoNcAPOvNVf .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-SdlMHOoNcAPOvNVf .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-SdlMHOoNcAPOvNVf .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-SdlMHOoNcAPOvNVf .marker{fill:#333333;stroke:#333333;}#mermaid-svg-SdlMHOoNcAPOvNVf .marker.cross{stroke:#333333;}#mermaid-svg-SdlMHOoNcAPOvNVf svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-SdlMHOoNcAPOvNVf p{margin:0;}#mermaid-svg-SdlMHOoNcAPOvNVf .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-SdlMHOoNcAPOvNVf .cluster-label text{fill:#333;}#mermaid-svg-SdlMHOoNcAPOvNVf .cluster-label span{color:#333;}#mermaid-svg-SdlMHOoNcAPOvNVf .cluster-label span p{background-color:transparent;}#mermaid-svg-SdlMHOoNcAPOvNVf .label text,#mermaid-svg-SdlMHOoNcAPOvNVf span{fill:#333;color:#333;}#mermaid-svg-SdlMHOoNcAPOvNVf .node rect,#mermaid-svg-SdlMHOoNcAPOvNVf .node circle,#mermaid-svg-SdlMHOoNcAPOvNVf .node ellipse,#mermaid-svg-SdlMHOoNcAPOvNVf .node polygon,#mermaid-svg-SdlMHOoNcAPOvNVf .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-SdlMHOoNcAPOvNVf .rough-node .label text,#mermaid-svg-SdlMHOoNcAPOvNVf .node .label text,#mermaid-svg-SdlMHOoNcAPOvNVf .image-shape .label,#mermaid-svg-SdlMHOoNcAPOvNVf .icon-shape .label{text-anchor:middle;}#mermaid-svg-SdlMHOoNcAPOvNVf .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-SdlMHOoNcAPOvNVf .rough-node .label,#mermaid-svg-SdlMHOoNcAPOvNVf .node .label,#mermaid-svg-SdlMHOoNcAPOvNVf .image-shape .label,#mermaid-svg-SdlMHOoNcAPOvNVf .icon-shape .label{text-align:center;}#mermaid-svg-SdlMHOoNcAPOvNVf .node.clickable{cursor:pointer;}#mermaid-svg-SdlMHOoNcAPOvNVf .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-SdlMHOoNcAPOvNVf .arrowheadPath{fill:#333333;}#mermaid-svg-SdlMHOoNcAPOvNVf .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-SdlMHOoNcAPOvNVf .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-SdlMHOoNcAPOvNVf .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-SdlMHOoNcAPOvNVf .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-SdlMHOoNcAPOvNVf .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-SdlMHOoNcAPOvNVf .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-SdlMHOoNcAPOvNVf .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-SdlMHOoNcAPOvNVf .cluster text{fill:#333;}#mermaid-svg-SdlMHOoNcAPOvNVf .cluster span{color:#333;}#mermaid-svg-SdlMHOoNcAPOvNVf div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-SdlMHOoNcAPOvNVf .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-SdlMHOoNcAPOvNVf rect.text{fill:none;stroke-width:0;}#mermaid-svg-SdlMHOoNcAPOvNVf .icon-shape,#mermaid-svg-SdlMHOoNcAPOvNVf .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-SdlMHOoNcAPOvNVf .icon-shape p,#mermaid-svg-SdlMHOoNcAPOvNVf .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-SdlMHOoNcAPOvNVf .icon-shape .label rect,#mermaid-svg-SdlMHOoNcAPOvNVf .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-SdlMHOoNcAPOvNVf .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-SdlMHOoNcAPOvNVf .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-SdlMHOoNcAPOvNVf :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 告警出现
确认时间和影响范围
查看近期发布和配置变更
查询日志和 traceId
查看监控指标
查看链路追踪
定位瓶颈层级
必要时使用 Arthas
制定止血方案
验证指标恢复
补充长期修复和复盘

具体动作可以按优先级来：

阶段	动作
止血	回滚、降级、限流、扩容、切流量
定位	日志、监控、链路追踪、Arthas、慢查询
修复	改代码、调配置、优化 SQL、补缓存、改重试
验证	看错误率、响应时间、QPS、业务成功率
复盘	补告警、补日志、补压测、补预案

生产排障的关键，不是会背多少工具，而是能不能沿着证据链把问题一步步压缩。

从"用户说慢"压缩到"某个接口慢"。

从"某个接口慢"压缩到"某个服务慢"。

从"某个服务慢"压缩到"某个方法、某条 SQL、某个下游调用慢"。

再从这个点拿出修复方案和验证结果。

这就是排障能力。