MySQL 数据库性能监控:基于 mysqld_exporter 监控连接数、QPS、慢查询及主从复制延迟
业务方投诉"数据库慢",你登上服务器一看------CPU 20%、内存 30%、磁盘没满。一切正常,但应用就是卡。因为你看的是"服务器指标",不是"数据库指标"。mysqld_exporter 告诉你 MySQL 到底在经历什么。
一、一次"服务器正常"的数据库故障
"数据库又超时了。"
老李打开监控:CPU 20%,内存 3GB/16GB,磁盘 IO 5%,网络流量正常。服务器看起来"健康得不能再健康"。
他登录 MySQL 执行 SHOW PROCESSLIST------结果被震住了:300 多个活跃连接,80% 处于 Waiting for table metadata lock 状态。再查慢查询日志,一条原本 20ms 的 UPDATE 语句,现在跑了 15 秒还没结束。
#mermaid-svg-HCMGTsvd16CpZ9i8{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-HCMGTsvd16CpZ9i8 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-HCMGTsvd16CpZ9i8 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-HCMGTsvd16CpZ9i8 .error-icon{fill:#552222;}#mermaid-svg-HCMGTsvd16CpZ9i8 .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-HCMGTsvd16CpZ9i8 .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-HCMGTsvd16CpZ9i8 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-HCMGTsvd16CpZ9i8 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-HCMGTsvd16CpZ9i8 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-HCMGTsvd16CpZ9i8 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-HCMGTsvd16CpZ9i8 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-HCMGTsvd16CpZ9i8 .marker{fill:#333333;stroke:#333333;}#mermaid-svg-HCMGTsvd16CpZ9i8 .marker.cross{stroke:#333333;}#mermaid-svg-HCMGTsvd16CpZ9i8 svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-HCMGTsvd16CpZ9i8 p{margin:0;}#mermaid-svg-HCMGTsvd16CpZ9i8 .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-HCMGTsvd16CpZ9i8 .cluster-label text{fill:#333;}#mermaid-svg-HCMGTsvd16CpZ9i8 .cluster-label span{color:#333;}#mermaid-svg-HCMGTsvd16CpZ9i8 .cluster-label span p{background-color:transparent;}#mermaid-svg-HCMGTsvd16CpZ9i8 .label text,#mermaid-svg-HCMGTsvd16CpZ9i8 span{fill:#333;color:#333;}#mermaid-svg-HCMGTsvd16CpZ9i8 .node rect,#mermaid-svg-HCMGTsvd16CpZ9i8 .node circle,#mermaid-svg-HCMGTsvd16CpZ9i8 .node ellipse,#mermaid-svg-HCMGTsvd16CpZ9i8 .node polygon,#mermaid-svg-HCMGTsvd16CpZ9i8 .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-HCMGTsvd16CpZ9i8 .rough-node .label text,#mermaid-svg-HCMGTsvd16CpZ9i8 .node .label text,#mermaid-svg-HCMGTsvd16CpZ9i8 .image-shape .label,#mermaid-svg-HCMGTsvd16CpZ9i8 .icon-shape .label{text-anchor:middle;}#mermaid-svg-HCMGTsvd16CpZ9i8 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-HCMGTsvd16CpZ9i8 .rough-node .label,#mermaid-svg-HCMGTsvd16CpZ9i8 .node .label,#mermaid-svg-HCMGTsvd16CpZ9i8 .image-shape .label,#mermaid-svg-HCMGTsvd16CpZ9i8 .icon-shape .label{text-align:center;}#mermaid-svg-HCMGTsvd16CpZ9i8 .node.clickable{cursor:pointer;}#mermaid-svg-HCMGTsvd16CpZ9i8 .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-HCMGTsvd16CpZ9i8 .arrowheadPath{fill:#333333;}#mermaid-svg-HCMGTsvd16CpZ9i8 .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-HCMGTsvd16CpZ9i8 .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-HCMGTsvd16CpZ9i8 .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-HCMGTsvd16CpZ9i8 .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-HCMGTsvd16CpZ9i8 .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-HCMGTsvd16CpZ9i8 .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-HCMGTsvd16CpZ9i8 .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-HCMGTsvd16CpZ9i8 .cluster text{fill:#333;}#mermaid-svg-HCMGTsvd16CpZ9i8 .cluster span{color:#333;}#mermaid-svg-HCMGTsvd16CpZ9i8 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-HCMGTsvd16CpZ9i8 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-HCMGTsvd16CpZ9i8 rect.text{fill:none;stroke-width:0;}#mermaid-svg-HCMGTsvd16CpZ9i8 .icon-shape,#mermaid-svg-HCMGTsvd16CpZ9i8 .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-HCMGTsvd16CpZ9i8 .icon-shape p,#mermaid-svg-HCMGTsvd16CpZ9i8 .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-HCMGTsvd16CpZ9i8 .icon-shape .label rect,#mermaid-svg-HCMGTsvd16CpZ9i8 .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-HCMGTsvd16CpZ9i8 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-HCMGTsvd16CpZ9i8 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-HCMGTsvd16CpZ9i8 :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} MySQL 内部(实际已崩)
服务器监控(看起来正常)
假象
假象
假象
CPU 20% ✅
内存 3GB/16GB ✅
磁盘 IO 5% ✅
300+ 活跃连接
80% 等待 metadata lock
一条 UPDATE 跑了 15 秒
问题找到了,但更扎心的是------为什么这些数据库指标没有人提前监控?
服务器监控显示"正常",是因为它在看操作系统,而不是 MySQL。CPU 20% 说明 MySQL 没跑满?错------是因为大量连接被阻塞等待,线程根本没在工作。
这就需要 mysqld_exporter。
二、mysqld_exporter 是什么?
mysqld_exporter 是 Prometheus 官方出品的 MySQL 监控工具,通过执行 SHOW STATUS、SHOW VARIABLES、SHOW SLAVE STATUS 等 SQL 语句,把 MySQL 的内部状态转换成 Prometheus 指标。
它能回答的四个核心问题
| 问题 | mysqld_exporter 指标 | 为什么重要 |
|---|---|---|
| 数据库有多少人在用? | mysql_global_status_threads_connected |
连接数超限会导致拒绝服务 |
| 每秒处理多少请求? | 从 mysql_global_status_queries 计算 rate() |
QPS 突降说明应用挂了,突升说明被攻击 |
| SQL 写得有多烂? | mysql_global_status_slow_queries |
慢查询侵蚀整体性能 |
| 主从同步还活着吗? | mysql_slave_status_seconds_behind_master |
主从延迟=读到旧数据 |
一句话价值:让 MySQL 从"黑盒"变成"透明盒"。
三、整体架构图

#mermaid-svg-8DgL39RCxrhiHv4M{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-8DgL39RCxrhiHv4M .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-8DgL39RCxrhiHv4M .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-8DgL39RCxrhiHv4M .error-icon{fill:#552222;}#mermaid-svg-8DgL39RCxrhiHv4M .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-8DgL39RCxrhiHv4M .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-8DgL39RCxrhiHv4M .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-8DgL39RCxrhiHv4M .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-8DgL39RCxrhiHv4M .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-8DgL39RCxrhiHv4M .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-8DgL39RCxrhiHv4M .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-8DgL39RCxrhiHv4M .marker{fill:#333333;stroke:#333333;}#mermaid-svg-8DgL39RCxrhiHv4M .marker.cross{stroke:#333333;}#mermaid-svg-8DgL39RCxrhiHv4M svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-8DgL39RCxrhiHv4M p{margin:0;}#mermaid-svg-8DgL39RCxrhiHv4M .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-8DgL39RCxrhiHv4M .cluster-label text{fill:#333;}#mermaid-svg-8DgL39RCxrhiHv4M .cluster-label span{color:#333;}#mermaid-svg-8DgL39RCxrhiHv4M .cluster-label span p{background-color:transparent;}#mermaid-svg-8DgL39RCxrhiHv4M .label text,#mermaid-svg-8DgL39RCxrhiHv4M span{fill:#333;color:#333;}#mermaid-svg-8DgL39RCxrhiHv4M .node rect,#mermaid-svg-8DgL39RCxrhiHv4M .node circle,#mermaid-svg-8DgL39RCxrhiHv4M .node ellipse,#mermaid-svg-8DgL39RCxrhiHv4M .node polygon,#mermaid-svg-8DgL39RCxrhiHv4M .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-8DgL39RCxrhiHv4M .rough-node .label text,#mermaid-svg-8DgL39RCxrhiHv4M .node .label text,#mermaid-svg-8DgL39RCxrhiHv4M .image-shape .label,#mermaid-svg-8DgL39RCxrhiHv4M .icon-shape .label{text-anchor:middle;}#mermaid-svg-8DgL39RCxrhiHv4M .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-8DgL39RCxrhiHv4M .rough-node .label,#mermaid-svg-8DgL39RCxrhiHv4M .node .label,#mermaid-svg-8DgL39RCxrhiHv4M .image-shape .label,#mermaid-svg-8DgL39RCxrhiHv4M .icon-shape .label{text-align:center;}#mermaid-svg-8DgL39RCxrhiHv4M .node.clickable{cursor:pointer;}#mermaid-svg-8DgL39RCxrhiHv4M .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-8DgL39RCxrhiHv4M .arrowheadPath{fill:#333333;}#mermaid-svg-8DgL39RCxrhiHv4M .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-8DgL39RCxrhiHv4M .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-8DgL39RCxrhiHv4M .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-8DgL39RCxrhiHv4M .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-8DgL39RCxrhiHv4M .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-8DgL39RCxrhiHv4M .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-8DgL39RCxrhiHv4M .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-8DgL39RCxrhiHv4M .cluster text{fill:#333;}#mermaid-svg-8DgL39RCxrhiHv4M .cluster span{color:#333;}#mermaid-svg-8DgL39RCxrhiHv4M div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-8DgL39RCxrhiHv4M .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-8DgL39RCxrhiHv4M rect.text{fill:none;stroke-width:0;}#mermaid-svg-8DgL39RCxrhiHv4M .icon-shape,#mermaid-svg-8DgL39RCxrhiHv4M .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-8DgL39RCxrhiHv4M .icon-shape p,#mermaid-svg-8DgL39RCxrhiHv4M .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-8DgL39RCxrhiHv4M .icon-shape .label rect,#mermaid-svg-8DgL39RCxrhiHv4M .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-8DgL39RCxrhiHv4M .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-8DgL39RCxrhiHv4M .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-8DgL39RCxrhiHv4M :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 可视化
存储与告警
采集层
MySQL 实例
MySQL Master
主库
MySQL Slave
从库
mysqld_exporter
主库采集
mysqld_exporter
从库采集
Prometheus
指标存储 + 告警规则
Grafana 仪表盘
Dashboard ID: 7362
告警通知
- 蓝色 :MySQL 实例和
mysqld_exporter------每个数据库实例配一个 Exporter,负责采集内部指标 - 橙色:Prometheus 采集层
- 绿色:展示和告警层
注意:主从实例需要分别部署 Exporter,它们的指标各有侧重。
四、核心指标详解
4.1 连接数监控
连接数是最直接的"压力表"------连接数高不一定有问题,但连接数耗尽一定是灾难。
#mermaid-svg-oxdHWlm0FobX0NCc{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-oxdHWlm0FobX0NCc .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-oxdHWlm0FobX0NCc .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-oxdHWlm0FobX0NCc .error-icon{fill:#552222;}#mermaid-svg-oxdHWlm0FobX0NCc .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-oxdHWlm0FobX0NCc .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-oxdHWlm0FobX0NCc .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-oxdHWlm0FobX0NCc .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-oxdHWlm0FobX0NCc .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-oxdHWlm0FobX0NCc .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-oxdHWlm0FobX0NCc .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-oxdHWlm0FobX0NCc .marker{fill:#333333;stroke:#333333;}#mermaid-svg-oxdHWlm0FobX0NCc .marker.cross{stroke:#333333;}#mermaid-svg-oxdHWlm0FobX0NCc svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-oxdHWlm0FobX0NCc p{margin:0;}#mermaid-svg-oxdHWlm0FobX0NCc .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-oxdHWlm0FobX0NCc .cluster-label text{fill:#333;}#mermaid-svg-oxdHWlm0FobX0NCc .cluster-label span{color:#333;}#mermaid-svg-oxdHWlm0FobX0NCc .cluster-label span p{background-color:transparent;}#mermaid-svg-oxdHWlm0FobX0NCc .label text,#mermaid-svg-oxdHWlm0FobX0NCc span{fill:#333;color:#333;}#mermaid-svg-oxdHWlm0FobX0NCc .node rect,#mermaid-svg-oxdHWlm0FobX0NCc .node circle,#mermaid-svg-oxdHWlm0FobX0NCc .node ellipse,#mermaid-svg-oxdHWlm0FobX0NCc .node polygon,#mermaid-svg-oxdHWlm0FobX0NCc .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-oxdHWlm0FobX0NCc .rough-node .label text,#mermaid-svg-oxdHWlm0FobX0NCc .node .label text,#mermaid-svg-oxdHWlm0FobX0NCc .image-shape .label,#mermaid-svg-oxdHWlm0FobX0NCc .icon-shape .label{text-anchor:middle;}#mermaid-svg-oxdHWlm0FobX0NCc .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-oxdHWlm0FobX0NCc .rough-node .label,#mermaid-svg-oxdHWlm0FobX0NCc .node .label,#mermaid-svg-oxdHWlm0FobX0NCc .image-shape .label,#mermaid-svg-oxdHWlm0FobX0NCc .icon-shape .label{text-align:center;}#mermaid-svg-oxdHWlm0FobX0NCc .node.clickable{cursor:pointer;}#mermaid-svg-oxdHWlm0FobX0NCc .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-oxdHWlm0FobX0NCc .arrowheadPath{fill:#333333;}#mermaid-svg-oxdHWlm0FobX0NCc .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-oxdHWlm0FobX0NCc .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-oxdHWlm0FobX0NCc .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-oxdHWlm0FobX0NCc .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-oxdHWlm0FobX0NCc .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-oxdHWlm0FobX0NCc .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-oxdHWlm0FobX0NCc .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-oxdHWlm0FobX0NCc .cluster text{fill:#333;}#mermaid-svg-oxdHWlm0FobX0NCc .cluster span{color:#333;}#mermaid-svg-oxdHWlm0FobX0NCc div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-oxdHWlm0FobX0NCc .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-oxdHWlm0FobX0NCc rect.text{fill:none;stroke-width:0;}#mermaid-svg-oxdHWlm0FobX0NCc .icon-shape,#mermaid-svg-oxdHWlm0FobX0NCc .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-oxdHWlm0FobX0NCc .icon-shape p,#mermaid-svg-oxdHWlm0FobX0NCc .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-oxdHWlm0FobX0NCc .icon-shape .label rect,#mermaid-svg-oxdHWlm0FobX0NCc .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-oxdHWlm0FobX0NCc .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-oxdHWlm0FobX0NCc .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-oxdHWlm0FobX0NCc :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 连接池状态
否
是
max_connections
最大连接数
threads_connected
已用连接数
threads_running
正在执行的连接
使用率 = connected / max_connections
使用率 > 80%?
✅ 正常
⚠️ 连接数告警
需扩容或排查泄漏
| 指标名 | 含义 | 正常值 | 告警阈值 |
|---|---|---|---|
mysql_global_status_threads_connected |
当前连接数 | 低于 max_connections 的 80% | > max_connections × 0.8 |
mysql_global_status_threads_running |
正在执行的连接 | 越低越好,高说明阻塞 | > 50% 的 connected |
mysql_global_variables_max_connections |
最大允许连接数 | 根据规格配置 | 配置不足时需要调大 |
PromQL 示例:连接数使用率
promql
mysql_global_status_threads_connected / mysql_global_variables_max_connections
4.2 QPS/TPS 监控
QPS 是数据库吞吐量的核心指标。
#mermaid-svg-ixpWApBvyPZQCiwf{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-ixpWApBvyPZQCiwf .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-ixpWApBvyPZQCiwf .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-ixpWApBvyPZQCiwf .error-icon{fill:#552222;}#mermaid-svg-ixpWApBvyPZQCiwf .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-ixpWApBvyPZQCiwf .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-ixpWApBvyPZQCiwf .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-ixpWApBvyPZQCiwf .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-ixpWApBvyPZQCiwf .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-ixpWApBvyPZQCiwf .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-ixpWApBvyPZQCiwf .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-ixpWApBvyPZQCiwf .marker{fill:#333333;stroke:#333333;}#mermaid-svg-ixpWApBvyPZQCiwf .marker.cross{stroke:#333333;}#mermaid-svg-ixpWApBvyPZQCiwf svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-ixpWApBvyPZQCiwf p{margin:0;}#mermaid-svg-ixpWApBvyPZQCiwf .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-ixpWApBvyPZQCiwf .cluster-label text{fill:#333;}#mermaid-svg-ixpWApBvyPZQCiwf .cluster-label span{color:#333;}#mermaid-svg-ixpWApBvyPZQCiwf .cluster-label span p{background-color:transparent;}#mermaid-svg-ixpWApBvyPZQCiwf .label text,#mermaid-svg-ixpWApBvyPZQCiwf span{fill:#333;color:#333;}#mermaid-svg-ixpWApBvyPZQCiwf .node rect,#mermaid-svg-ixpWApBvyPZQCiwf .node circle,#mermaid-svg-ixpWApBvyPZQCiwf .node ellipse,#mermaid-svg-ixpWApBvyPZQCiwf .node polygon,#mermaid-svg-ixpWApBvyPZQCiwf .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-ixpWApBvyPZQCiwf .rough-node .label text,#mermaid-svg-ixpWApBvyPZQCiwf .node .label text,#mermaid-svg-ixpWApBvyPZQCiwf .image-shape .label,#mermaid-svg-ixpWApBvyPZQCiwf .icon-shape .label{text-anchor:middle;}#mermaid-svg-ixpWApBvyPZQCiwf .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-ixpWApBvyPZQCiwf .rough-node .label,#mermaid-svg-ixpWApBvyPZQCiwf .node .label,#mermaid-svg-ixpWApBvyPZQCiwf .image-shape .label,#mermaid-svg-ixpWApBvyPZQCiwf .icon-shape .label{text-align:center;}#mermaid-svg-ixpWApBvyPZQCiwf .node.clickable{cursor:pointer;}#mermaid-svg-ixpWApBvyPZQCiwf .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-ixpWApBvyPZQCiwf .arrowheadPath{fill:#333333;}#mermaid-svg-ixpWApBvyPZQCiwf .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-ixpWApBvyPZQCiwf .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-ixpWApBvyPZQCiwf .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-ixpWApBvyPZQCiwf .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-ixpWApBvyPZQCiwf .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-ixpWApBvyPZQCiwf .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-ixpWApBvyPZQCiwf .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-ixpWApBvyPZQCiwf .cluster text{fill:#333;}#mermaid-svg-ixpWApBvyPZQCiwf .cluster span{color:#333;}#mermaid-svg-ixpWApBvyPZQCiwf div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-ixpWApBvyPZQCiwf .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-ixpWApBvyPZQCiwf rect.text{fill:none;stroke-width:0;}#mermaid-svg-ixpWApBvyPZQCiwf .icon-shape,#mermaid-svg-ixpWApBvyPZQCiwf .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-ixpWApBvyPZQCiwf .icon-shape p,#mermaid-svg-ixpWApBvyPZQCiwf .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-ixpWApBvyPZQCiwf .icon-shape .label rect,#mermaid-svg-ixpWApBvyPZQCiwf .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-ixpWApBvyPZQCiwf .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-ixpWApBvyPZQCiwf .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-ixpWApBvyPZQCiwf :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 是
否
是
否
是
否
QPS 变化
QPS 突降?
检查上游应用是否挂了
QPS 突升?
检查是否被攻击或代码变更
QPS 正常但业务慢?
检查慢查询
✅ 正常
promql
# QPS(每秒查询数)
rate(mysql_global_status_queries[1m])
# TPS(每秒事务数,针对 InnoDB)
rate(mysql_global_status_innodb_row_lock_waits[1m])
趋势解读:
| 现象 | 可能原因 |
|---|---|
| QPS 突降 | 上游应用可能挂了 |
| QPS 突升 | 被攻击,或代码逻辑变化产生大量查询 |
| QPS 正常但业务慢 | 单条查询变慢了(看慢查询) |
4.3 慢查询监控
promql
# 慢查询速率(过去5分钟)
rate(mysql_global_status_slow_queries[5m])
# 慢查询占总查询的比例
mysql_global_status_slow_queries / mysql_global_status_queries
经验值:慢查询占比超过 0.1%(千分之一)就需要关注,超过 1% 就是严重问题。
4.4 InnoDB 行锁与缓冲池
行锁等待是"数据库慢"的常见原因。
#mermaid-svg-8CGS51nR5uDcKWW4{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-8CGS51nR5uDcKWW4 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-8CGS51nR5uDcKWW4 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-8CGS51nR5uDcKWW4 .error-icon{fill:#552222;}#mermaid-svg-8CGS51nR5uDcKWW4 .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-8CGS51nR5uDcKWW4 .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-8CGS51nR5uDcKWW4 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-8CGS51nR5uDcKWW4 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-8CGS51nR5uDcKWW4 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-8CGS51nR5uDcKWW4 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-8CGS51nR5uDcKWW4 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-8CGS51nR5uDcKWW4 .marker{fill:#333333;stroke:#333333;}#mermaid-svg-8CGS51nR5uDcKWW4 .marker.cross{stroke:#333333;}#mermaid-svg-8CGS51nR5uDcKWW4 svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-8CGS51nR5uDcKWW4 p{margin:0;}#mermaid-svg-8CGS51nR5uDcKWW4 .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-8CGS51nR5uDcKWW4 .cluster-label text{fill:#333;}#mermaid-svg-8CGS51nR5uDcKWW4 .cluster-label span{color:#333;}#mermaid-svg-8CGS51nR5uDcKWW4 .cluster-label span p{background-color:transparent;}#mermaid-svg-8CGS51nR5uDcKWW4 .label text,#mermaid-svg-8CGS51nR5uDcKWW4 span{fill:#333;color:#333;}#mermaid-svg-8CGS51nR5uDcKWW4 .node rect,#mermaid-svg-8CGS51nR5uDcKWW4 .node circle,#mermaid-svg-8CGS51nR5uDcKWW4 .node ellipse,#mermaid-svg-8CGS51nR5uDcKWW4 .node polygon,#mermaid-svg-8CGS51nR5uDcKWW4 .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-8CGS51nR5uDcKWW4 .rough-node .label text,#mermaid-svg-8CGS51nR5uDcKWW4 .node .label text,#mermaid-svg-8CGS51nR5uDcKWW4 .image-shape .label,#mermaid-svg-8CGS51nR5uDcKWW4 .icon-shape .label{text-anchor:middle;}#mermaid-svg-8CGS51nR5uDcKWW4 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-8CGS51nR5uDcKWW4 .rough-node .label,#mermaid-svg-8CGS51nR5uDcKWW4 .node .label,#mermaid-svg-8CGS51nR5uDcKWW4 .image-shape .label,#mermaid-svg-8CGS51nR5uDcKWW4 .icon-shape .label{text-align:center;}#mermaid-svg-8CGS51nR5uDcKWW4 .node.clickable{cursor:pointer;}#mermaid-svg-8CGS51nR5uDcKWW4 .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-8CGS51nR5uDcKWW4 .arrowheadPath{fill:#333333;}#mermaid-svg-8CGS51nR5uDcKWW4 .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-8CGS51nR5uDcKWW4 .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-8CGS51nR5uDcKWW4 .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-8CGS51nR5uDcKWW4 .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-8CGS51nR5uDcKWW4 .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-8CGS51nR5uDcKWW4 .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-8CGS51nR5uDcKWW4 .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-8CGS51nR5uDcKWW4 .cluster text{fill:#333;}#mermaid-svg-8CGS51nR5uDcKWW4 .cluster span{color:#333;}#mermaid-svg-8CGS51nR5uDcKWW4 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-8CGS51nR5uDcKWW4 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-8CGS51nR5uDcKWW4 rect.text{fill:none;stroke-width:0;}#mermaid-svg-8CGS51nR5uDcKWW4 .icon-shape,#mermaid-svg-8CGS51nR5uDcKWW4 .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-8CGS51nR5uDcKWW4 .icon-shape p,#mermaid-svg-8CGS51nR5uDcKWW4 .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-8CGS51nR5uDcKWW4 .icon-shape .label rect,#mermaid-svg-8CGS51nR5uDcKWW4 .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-8CGS51nR5uDcKWW4 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-8CGS51nR5uDcKWW4 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-8CGS51nR5uDcKWW4 :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 缓冲池命中率
否
是
read_requests
总读请求
buffer_pool_reads
从磁盘读取
命中率 = (1 - reads/requests) × 100
命中率 < 99%?
✅ 命中率正常
⬆️ 增加 buffer pool 大小
| 指标 | 含义 | 高值说明 |
|---|---|---|
mysql_global_status_innodb_row_lock_waits |
行锁等待次数 | 事务设计有问题,或同一行被频繁争用 |
mysql_global_status_innodb_row_lock_time |
行锁总等待时间 | 单位微秒,除以等待次数得平均耗时 |
mysql_global_status_innodb_buffer_pool_reads |
从磁盘读的页数 | 高说明 buffer pool 太小 |
mysql_global_status_innodb_buffer_pool_read_requests |
总读请求 | 用 reads/requests 得命中率 |
PromQL 示例:InnoDB 缓冲池命中率
promql
(1 - (mysql_global_status_innodb_buffer_pool_reads / mysql_global_status_innodb_buffer_pool_read_requests)) * 100
命中率低于 99% 就该考虑增加 buffer pool 大小了。
4.5 主从复制延迟------最容易被忽略
主从延迟会导致"刚写入的数据读不到",是微服务架构里的隐形杀手。
#mermaid-svg-NdJdLsMn4w5Yupgx{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-NdJdLsMn4w5Yupgx .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-NdJdLsMn4w5Yupgx .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-NdJdLsMn4w5Yupgx .error-icon{fill:#552222;}#mermaid-svg-NdJdLsMn4w5Yupgx .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-NdJdLsMn4w5Yupgx .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-NdJdLsMn4w5Yupgx .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-NdJdLsMn4w5Yupgx .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-NdJdLsMn4w5Yupgx .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-NdJdLsMn4w5Yupgx .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-NdJdLsMn4w5Yupgx .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-NdJdLsMn4w5Yupgx .marker{fill:#333333;stroke:#333333;}#mermaid-svg-NdJdLsMn4w5Yupgx .marker.cross{stroke:#333333;}#mermaid-svg-NdJdLsMn4w5Yupgx svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-NdJdLsMn4w5Yupgx p{margin:0;}#mermaid-svg-NdJdLsMn4w5Yupgx .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-NdJdLsMn4w5Yupgx .cluster-label text{fill:#333;}#mermaid-svg-NdJdLsMn4w5Yupgx .cluster-label span{color:#333;}#mermaid-svg-NdJdLsMn4w5Yupgx .cluster-label span p{background-color:transparent;}#mermaid-svg-NdJdLsMn4w5Yupgx .label text,#mermaid-svg-NdJdLsMn4w5Yupgx span{fill:#333;color:#333;}#mermaid-svg-NdJdLsMn4w5Yupgx .node rect,#mermaid-svg-NdJdLsMn4w5Yupgx .node circle,#mermaid-svg-NdJdLsMn4w5Yupgx .node ellipse,#mermaid-svg-NdJdLsMn4w5Yupgx .node polygon,#mermaid-svg-NdJdLsMn4w5Yupgx .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-NdJdLsMn4w5Yupgx .rough-node .label text,#mermaid-svg-NdJdLsMn4w5Yupgx .node .label text,#mermaid-svg-NdJdLsMn4w5Yupgx .image-shape .label,#mermaid-svg-NdJdLsMn4w5Yupgx .icon-shape .label{text-anchor:middle;}#mermaid-svg-NdJdLsMn4w5Yupgx .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-NdJdLsMn4w5Yupgx .rough-node .label,#mermaid-svg-NdJdLsMn4w5Yupgx .node .label,#mermaid-svg-NdJdLsMn4w5Yupgx .image-shape .label,#mermaid-svg-NdJdLsMn4w5Yupgx .icon-shape .label{text-align:center;}#mermaid-svg-NdJdLsMn4w5Yupgx .node.clickable{cursor:pointer;}#mermaid-svg-NdJdLsMn4w5Yupgx .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-NdJdLsMn4w5Yupgx .arrowheadPath{fill:#333333;}#mermaid-svg-NdJdLsMn4w5Yupgx .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-NdJdLsMn4w5Yupgx .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-NdJdLsMn4w5Yupgx .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-NdJdLsMn4w5Yupgx .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-NdJdLsMn4w5Yupgx .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-NdJdLsMn4w5Yupgx .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-NdJdLsMn4w5Yupgx .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-NdJdLsMn4w5Yupgx .cluster text{fill:#333;}#mermaid-svg-NdJdLsMn4w5Yupgx .cluster span{color:#333;}#mermaid-svg-NdJdLsMn4w5Yupgx div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-NdJdLsMn4w5Yupgx .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-NdJdLsMn4w5Yupgx rect.text{fill:none;stroke-width:0;}#mermaid-svg-NdJdLsMn4w5Yupgx .icon-shape,#mermaid-svg-NdJdLsMn4w5Yupgx .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-NdJdLsMn4w5Yupgx .icon-shape p,#mermaid-svg-NdJdLsMn4w5Yupgx .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-NdJdLsMn4w5Yupgx .icon-shape .label rect,#mermaid-svg-NdJdLsMn4w5Yupgx .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-NdJdLsMn4w5Yupgx .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-NdJdLsMn4w5Yupgx .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-NdJdLsMn4w5Yupgx :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 监控指标
复制链路
主从复制架构
MySQL Master
主库
MySQL Slave
从库
binlog
写入
IO 线程
拉取 binlog
relay log
中继日志
SQL 线程
回放
seconds_behind_master
延迟秒数
slave_io_running
IO 线程状态
slave_sql_running
SQL 线程状态
| 指标 | 含义 | 正常值 | 告警阈值 |
|---|---|---|---|
mysql_slave_status_seconds_behind_master |
延迟秒数 | 0 | > 10 秒,持续 5 分钟 |
mysql_slave_status_slave_sql_running |
SQL 线程状态 | 1 | 不为 1 说明同步中断 |
mysql_slave_status_slave_io_running |
IO 线程状态 | 1 | 不为 1 说明拉取 binlog 失败 |
注意:
seconds_behind_master=0不代表没有延迟------如果主库没写入,延迟就是 0,但 IO 线程可能已经断了。所以必须同时监控两个 Running 状态。
五、实战:部署与配置
5.1 创建 MySQL 监控用户
sql
CREATE USER 'mysqld_exporter'@'%' IDENTIFIED BY 'strong_password';
GRANT PROCESS, REPLICATION CLIENT, SELECT ON *.* TO 'mysqld_exporter'@'%';
FLUSH PRIVILEGES;
5.2 部署 mysqld_exporter(Docker 方式)
bash
docker run -d \
--name mysqld_exporter \
-p 9104:9104 \
-e DATA_SOURCE_NAME="mysqld_exporter:strong_password@(mysql-master:3306)/" \
prom/mysqld-exporter:latest
5.3 K8s 部署(Deployment + Service)
yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: mysqld-exporter-master
namespace: monitoring
spec:
replicas: 1
selector:
matchLabels:
app: mysqld-exporter
instance: master
template:
metadata:
labels:
app: mysqld-exporter
instance: master
spec:
containers:
- name: exporter
image: prom/mysqld-exporter:latest
args:
- --collect.info_schema.processlist
- --collect.info_schema.innodb_tablespaces
- --collect.perf_schema.table_lock_waits
env:
- name: DATA_SOURCE_NAME
value: "mysqld_exporter:password@(mysql-master:3306)/"
ports:
- containerPort: 9104
5.4 Prometheus 采集配置(ServiceMonitor)
yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: mysql-monitor
spec:
selector:
matchLabels:
app: mysqld-exporter
endpoints:
- port: metrics
interval: 15s
namespaceSelector:
matchNames:
- monitoring
5.5 Grafana 导入仪表盘
Grafana 官方 MySQL 仪表盘 ID:7362 (最流行)或 6239。
登录 Grafana → + → Import
输入 Dashboard ID:7362
选择 Prometheus 数据源
六、告警规则配置
#mermaid-svg-WBf68pCDlKsg3eey{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-WBf68pCDlKsg3eey .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-WBf68pCDlKsg3eey .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-WBf68pCDlKsg3eey .error-icon{fill:#552222;}#mermaid-svg-WBf68pCDlKsg3eey .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-WBf68pCDlKsg3eey .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-WBf68pCDlKsg3eey .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-WBf68pCDlKsg3eey .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-WBf68pCDlKsg3eey .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-WBf68pCDlKsg3eey .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-WBf68pCDlKsg3eey .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-WBf68pCDlKsg3eey .marker{fill:#333333;stroke:#333333;}#mermaid-svg-WBf68pCDlKsg3eey .marker.cross{stroke:#333333;}#mermaid-svg-WBf68pCDlKsg3eey svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-WBf68pCDlKsg3eey p{margin:0;}#mermaid-svg-WBf68pCDlKsg3eey .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-WBf68pCDlKsg3eey .cluster-label text{fill:#333;}#mermaid-svg-WBf68pCDlKsg3eey .cluster-label span{color:#333;}#mermaid-svg-WBf68pCDlKsg3eey .cluster-label span p{background-color:transparent;}#mermaid-svg-WBf68pCDlKsg3eey .label text,#mermaid-svg-WBf68pCDlKsg3eey span{fill:#333;color:#333;}#mermaid-svg-WBf68pCDlKsg3eey .node rect,#mermaid-svg-WBf68pCDlKsg3eey .node circle,#mermaid-svg-WBf68pCDlKsg3eey .node ellipse,#mermaid-svg-WBf68pCDlKsg3eey .node polygon,#mermaid-svg-WBf68pCDlKsg3eey .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-WBf68pCDlKsg3eey .rough-node .label text,#mermaid-svg-WBf68pCDlKsg3eey .node .label text,#mermaid-svg-WBf68pCDlKsg3eey .image-shape .label,#mermaid-svg-WBf68pCDlKsg3eey .icon-shape .label{text-anchor:middle;}#mermaid-svg-WBf68pCDlKsg3eey .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-WBf68pCDlKsg3eey .rough-node .label,#mermaid-svg-WBf68pCDlKsg3eey .node .label,#mermaid-svg-WBf68pCDlKsg3eey .image-shape .label,#mermaid-svg-WBf68pCDlKsg3eey .icon-shape .label{text-align:center;}#mermaid-svg-WBf68pCDlKsg3eey .node.clickable{cursor:pointer;}#mermaid-svg-WBf68pCDlKsg3eey .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-WBf68pCDlKsg3eey .arrowheadPath{fill:#333333;}#mermaid-svg-WBf68pCDlKsg3eey .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-WBf68pCDlKsg3eey .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-WBf68pCDlKsg3eey .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-WBf68pCDlKsg3eey .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-WBf68pCDlKsg3eey .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-WBf68pCDlKsg3eey .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-WBf68pCDlKsg3eey .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-WBf68pCDlKsg3eey .cluster text{fill:#333;}#mermaid-svg-WBf68pCDlKsg3eey .cluster span{color:#333;}#mermaid-svg-WBf68pCDlKsg3eey div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-WBf68pCDlKsg3eey .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-WBf68pCDlKsg3eey rect.text{fill:none;stroke-width:0;}#mermaid-svg-WBf68pCDlKsg3eey .icon-shape,#mermaid-svg-WBf68pCDlKsg3eey .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-WBf68pCDlKsg3eey .icon-shape p,#mermaid-svg-WBf68pCDlKsg3eey .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-WBf68pCDlKsg3eey .icon-shape .label rect,#mermaid-svg-WBf68pCDlKsg3eey .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-WBf68pCDlKsg3eey .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-WBf68pCDlKsg3eey .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-WBf68pCDlKsg3eey :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 是持续5分钟
否
是持续2分钟
否
是持续5分钟
否
是持续1分钟
否
是持续10分钟
否
是持续5分钟
否
Prometheus 告警评估
连接数使用率 > 80%?
🔔 MySQL_High_Connection_Count
慢查询速率 > 10/s?
🔔 MySQL_High_Slow_Queries
主从延迟 > 10秒?
🔔 MySQL_Replication_Delay
同步线程异常?
🔔 MySQL_Slave_Not_Running
缓冲池命中率 < 99%?
🔔 MySQL_Low_Buffer_Pool_Hit
行锁等待 > 10/s?
🔔 MySQL_High_Row_Lock_Waits
✅ 数据库健康
yaml
groups:
- name: mysql-alerts
rules:
# 连接数超过最大连接的80%
- alert: MySQL_High_Connection_Count
expr: mysql_global_status_threads_connected / mysql_global_variables_max_connections > 0.8
for: 5m
annotations:
summary: "连接数过高 {{ $value | humanizePercentage }}"
# 慢查询速率过高
- alert: MySQL_High_Slow_Queries
expr: rate(mysql_global_status_slow_queries[5m]) > 10
for: 2m
annotations:
summary: "慢查询过多,当前速率 {{ $value }}/s"
# 主从延迟超过10秒
- alert: MySQL_Replication_Delay
expr: mysql_slave_status_seconds_behind_master > 10
for: 5m
annotations:
summary: "主从延迟 {{ $value }}秒"
# 主从同步线程中断
- alert: MySQL_Slave_Not_Running
expr: mysql_slave_status_slave_sql_running != 1 or mysql_slave_status_slave_io_running != 1
for: 1m
annotations:
summary: "主从同步线程异常"
# InnoDB缓冲池命中率低于99%
- alert: MySQL_Low_Buffer_Pool_Hit
expr: |
(1 - (mysql_global_status_innodb_buffer_pool_reads /
mysql_global_status_innodb_buffer_pool_read_requests)) * 100 < 99
for: 10m
annotations:
summary: "缓冲池命中率 {{ $value }}%,考虑增加内存"
# 行锁等待过多
- alert: MySQL_High_Row_Lock_Waits
expr: rate(mysql_global_status_innodb_row_lock_waits[5m]) > 10
for: 5m
annotations:
summary: "行锁等待过多,检查事务设计"
七、总结
MySQL 监控和服务器监控是两回事。
服务器 CPU 20%,MySQL 可能已经挂了------因为连接都在等锁,线程不工作,CPU 自然低。
mysqld_exporter 的价值:把 MySQL 的"内出血"指标暴露出来。
#mermaid-svg-7Le2zkDqjGkb76Ua{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-7Le2zkDqjGkb76Ua .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-7Le2zkDqjGkb76Ua .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-7Le2zkDqjGkb76Ua .error-icon{fill:#552222;}#mermaid-svg-7Le2zkDqjGkb76Ua .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-7Le2zkDqjGkb76Ua .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-7Le2zkDqjGkb76Ua .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-7Le2zkDqjGkb76Ua .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-7Le2zkDqjGkb76Ua .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-7Le2zkDqjGkb76Ua .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-7Le2zkDqjGkb76Ua .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-7Le2zkDqjGkb76Ua .marker{fill:#333333;stroke:#333333;}#mermaid-svg-7Le2zkDqjGkb76Ua .marker.cross{stroke:#333333;}#mermaid-svg-7Le2zkDqjGkb76Ua svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-7Le2zkDqjGkb76Ua p{margin:0;}#mermaid-svg-7Le2zkDqjGkb76Ua .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-7Le2zkDqjGkb76Ua .cluster-label text{fill:#333;}#mermaid-svg-7Le2zkDqjGkb76Ua .cluster-label span{color:#333;}#mermaid-svg-7Le2zkDqjGkb76Ua .cluster-label span p{background-color:transparent;}#mermaid-svg-7Le2zkDqjGkb76Ua .label text,#mermaid-svg-7Le2zkDqjGkb76Ua span{fill:#333;color:#333;}#mermaid-svg-7Le2zkDqjGkb76Ua .node rect,#mermaid-svg-7Le2zkDqjGkb76Ua .node circle,#mermaid-svg-7Le2zkDqjGkb76Ua .node ellipse,#mermaid-svg-7Le2zkDqjGkb76Ua .node polygon,#mermaid-svg-7Le2zkDqjGkb76Ua .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-7Le2zkDqjGkb76Ua .rough-node .label text,#mermaid-svg-7Le2zkDqjGkb76Ua .node .label text,#mermaid-svg-7Le2zkDqjGkb76Ua .image-shape .label,#mermaid-svg-7Le2zkDqjGkb76Ua .icon-shape .label{text-anchor:middle;}#mermaid-svg-7Le2zkDqjGkb76Ua .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-7Le2zkDqjGkb76Ua .rough-node .label,#mermaid-svg-7Le2zkDqjGkb76Ua .node .label,#mermaid-svg-7Le2zkDqjGkb76Ua .image-shape .label,#mermaid-svg-7Le2zkDqjGkb76Ua .icon-shape .label{text-align:center;}#mermaid-svg-7Le2zkDqjGkb76Ua .node.clickable{cursor:pointer;}#mermaid-svg-7Le2zkDqjGkb76Ua .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-7Le2zkDqjGkb76Ua .arrowheadPath{fill:#333333;}#mermaid-svg-7Le2zkDqjGkb76Ua .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-7Le2zkDqjGkb76Ua .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-7Le2zkDqjGkb76Ua .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-7Le2zkDqjGkb76Ua .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-7Le2zkDqjGkb76Ua .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-7Le2zkDqjGkb76Ua .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-7Le2zkDqjGkb76Ua .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-7Le2zkDqjGkb76Ua .cluster text{fill:#333;}#mermaid-svg-7Le2zkDqjGkb76Ua .cluster span{color:#333;}#mermaid-svg-7Le2zkDqjGkb76Ua div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-7Le2zkDqjGkb76Ua .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-7Le2zkDqjGkb76Ua rect.text{fill:none;stroke-width:0;}#mermaid-svg-7Le2zkDqjGkb76Ua .icon-shape,#mermaid-svg-7Le2zkDqjGkb76Ua .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-7Le2zkDqjGkb76Ua .icon-shape p,#mermaid-svg-7Le2zkDqjGkb76Ua .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-7Le2zkDqjGkb76Ua .icon-shape .label rect,#mermaid-svg-7Le2zkDqjGkb76Ua .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-7Le2zkDqjGkb76Ua .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-7Le2zkDqjGkb76Ua .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-7Le2zkDqjGkb76Ua :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 四个最该监控的信号
① 连接数使用率 > 80%
快要拒绝服务
② 慢查询速率 > 10/s
SQL 需要优化
③ 主从延迟 > 10秒
业务读到旧数据
④ 缓冲池命中率 < 99%
内存不足
扩容 / 排查连接泄漏
优化索引 / 重写 SQL
检查网络 / 从库性能
增加 buffer pool 内存
| 指标 | 告警条件 | 含义 |
|---|---|---|
| 线程连接数 / max_connections | > 0.8 | 连接数快满了,能赶在拒绝服务前扩容 |
rate(mysql_global_status_slow_queries[5m]) |
> 10 | SQL 写得有问题,需要优化索引或重写 |
mysql_slave_status_seconds_behind_master |
> 10 | 主从延迟,业务可能读到旧数据 |
1 - innodb_buffer_pool_reads/read_requests |
< 0.99 | 内存不足,需要给 MySQL 加内存 |
记住:数据库慢,别只看服务器------问
mysqld_exporter,它会告诉你真相。