🧰 第八章:Tomcat 调试与监控
目录
- [8.1 日志系统详解](#8.1 日志系统详解)
- [8.2 监控方式与工具](#8.2 监控方式与工具)
- [8.3 JMX 监控实现](#8.3 JMX 监控实现)
- [8.4 性能监控与诊断](#8.4 性能监控与诊断)
- [8.5 故障排查与调试](#8.5 故障排查与调试)
- [8.6 自动化监控方案](#8.6 自动化监控方案)
- [8.7 本章小结](#8.7 本章小结)
8.1 日志系统详解
8.1.1 日志文件结构
日志文件说明
| 日志文件 | 说明 | 位置 | 用途 |
|---|---|---|---|
| catalina.out | 主日志文件 | logs/ | 记录 Tomcat 启动、停止和错误信息 |
| catalina.yyyy-mm-dd.log | 按日期分割的日志 | logs/ | 每日的详细日志 |
| localhost.yyyy-mm-dd.log | 本地主机日志 | logs/ | 本地主机的详细日志 |
| manager.yyyy-mm-dd.log | 管理应用日志 | logs/ | Manager 应用的日志 |
| host-manager.yyyy-mm-dd.log | 主机管理日志 | logs/ | Host Manager 应用的日志 |
| localhost_access_log.yyyy-mm-dd.txt | 访问日志 | logs/ | HTTP 请求访问日志 |
日志配置
xml
<!-- 日志配置 -->
<Context path="/myapp" docBase="myapp">
<Valve className="org.apache.catalina.valves.AccessLogValve"
directory="logs"
prefix="localhost_access_log"
suffix=".txt"
pattern="%h %l %u %t "%r" %s %b %D"
resolveHosts="false" />
</Context>
8.1.2 日志级别配置
日志级别设置
properties
# logging.properties
# 根日志级别
.level = INFO
# 包级别日志
org.apache.catalina.level = INFO
org.apache.coyote.level = INFO
org.apache.tomcat.level = INFO
org.apache.catalina.startup.level = INFO
org.apache.catalina.core.level = INFO
org.apache.catalina.session.level = INFO
org.apache.catalina.connector.level = INFO
org.apache.catalina.valves.level = INFO
org.apache.catalina.authenticator.level = INFO
org.apache.catalina.realm.level = INFO
org.apache.catalina.loader.level = INFO
org.apache.catalina.util.level = INFO
org.apache.catalina.mbeans.level = INFO
org.apache.catalina.storeconfig.level = INFO
org.apache.catalina.ha.level = INFO
org.apache.catalina.tribes.level = INFO
org.apache.catalina.cluster.level = INFO
org.apache.catalina.ha.tcp.level = INFO
org.apache.catalina.ha.session.level = INFO
org.apache.catalina.ha.deploy.level = INFO
org.apache.catalina.ha.context.level = INFO
org.apache.catalina.ha.backup.level = INFO
org.apache.catalina.ha.channel.level = INFO
org.apache.catalina.ha.channel.sender.level = INFO
org.apache.catalina.ha.channel.receiver.level = INFO
org.apache.catalina.ha.channel.interceptor.level = INFO
org.apache.catalina.ha.channel.interceptor.tcp.level = INFO
org.apache.catalina.ha.channel.interceptor.tcp.TcpFailureDetector.level = INFO
org.apache.catalina.ha.channel.interceptor.tcp.TcpPingInterceptor.level = INFO
org.apache.catalina.ha.channel.interceptor.tcp.TcpValve.level = INFO
org.apache.catalina.ha.channel.interceptor.tcp.TcpFailureDetector.level = INFO
org.apache.catalina.ha.channel.interceptor.tcp.TcpPingInterceptor.level = INFO
org.apache.catalina.ha.channel.interceptor.tcp.TcpValve.level = INFO
8.1.3 自定义日志格式
访问日志格式
xml
<!-- 自定义访问日志格式 -->
<Valve className="org.apache.catalina.valves.AccessLogValve"
directory="logs"
prefix="localhost_access_log"
suffix=".txt"
pattern="%h %l %u %t "%r" %s %b %D %{User-Agent}i %{Referer}i"
resolveHosts="false" />
日志格式说明
| 格式符 | 说明 | 示例 |
|---|---|---|
| %h | 远程主机名或 IP | 192.168.1.100 |
| %l | 远程逻辑用户名 | - |
| %u | 远程用户 | admin |
| %t | 时间戳 | [25/Dec/2023:10:30:45 +0800] |
| %r | 请求行 | GET /myapp/servlet HTTP/1.1 |
| %s | 状态码 | 200 |
| %b | 响应字节数 | 1024 |
| %D | 处理时间(毫秒) | 150 |
| %{User-Agent}i | 用户代理 | Mozilla/5.0... |
| %{Referer}i | 引用页面 | http://example.com/ |
8.2 监控方式与工具
8.2.1 内置监控工具
Manager 应用监控
xml
<!-- 启用 Manager 应用 -->
<Context path="/manager" docBase="manager"
privileged="true" antiResourceLocking="false" antiJARLocking="false">
<Valve className="org.apache.catalina.valves.RemoteAddrValve"
allow="127\.0\.0\.1|::1|0:0:0:0:0:0:0:1" />
</Context>
Manager 功能
- 应用管理:部署、启动、停止、重新加载应用
- 会话管理:查看和管理用户会话
- 资源管理:查看系统资源使用情况
- 性能监控:查看请求统计和性能指标
8.2.2 外部监控工具
VisualVM 监控
java
// VisualVM 监控配置
public class VisualVMMonitor {
public void monitorTomcat() {
// 1. 获取 MBean 服务器
MBeanServer mBeanServer = ManagementFactory.getPlatformMBeanServer();
// 2. 监控线程池
try {
ObjectName threadPoolName = new ObjectName("Catalina:type=ThreadPool,name=*");
Set<ObjectName> threadPools = mBeanServer.queryNames(threadPoolName, null);
for (ObjectName threadPool : threadPools) {
int currentThreadCount = (Integer) mBeanServer.getAttribute(threadPool, "currentThreadCount");
int currentThreadsBusy = (Integer) mBeanServer.getAttribute(threadPool, "currentThreadsBusy");
int maxThreads = (Integer) mBeanServer.getAttribute(threadPool, "maxThreads");
System.out.println("Thread Pool: " + threadPool.getKeyProperty("name"));
System.out.println("Current Threads: " + currentThreadCount);
System.out.println("Busy Threads: " + currentThreadsBusy);
System.out.println("Max Threads: " + maxThreads);
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
JConsole 监控
java
// JConsole 监控配置
public class JConsoleMonitor {
public void monitorTomcat() {
// 1. 获取内存使用情况
MemoryMXBean memoryBean = ManagementFactory.getMemoryMXBean();
MemoryUsage heapUsage = memoryBean.getHeapMemoryUsage();
MemoryUsage nonHeapUsage = memoryBean.getNonHeapMemoryUsage();
System.out.println("Heap Memory: " + heapUsage.getUsed() / 1024 / 1024 + "MB");
System.out.println("Non-Heap Memory: " + nonHeapUsage.getUsed() / 1024 / 1024 + "MB");
// 2. 获取线程信息
ThreadMXBean threadBean = ManagementFactory.getThreadMXBean();
int threadCount = threadBean.getThreadCount();
int peakThreadCount = threadBean.getPeakThreadCount();
long totalStartedThreadCount = threadBean.getTotalStartedThreadCount();
System.out.println("Thread Count: " + threadCount);
System.out.println("Peak Thread Count: " + peakThreadCount);
System.out.println("Total Started Thread Count: " + totalStartedThreadCount);
// 3. 获取 GC 信息
List<GarbageCollectorMXBean> gcBeans = ManagementFactory.getGarbageCollectorMXBeans();
for (GarbageCollectorMXBean gcBean : gcBeans) {
System.out.println("GC Name: " + gcBean.getName());
System.out.println("GC Count: " + gcBean.getCollectionCount());
System.out.println("GC Time: " + gcBean.getCollectionTime() + "ms");
}
}
}
8.2.3 第三方监控工具
Prometheus + Grafana 监控
yaml
# prometheus.yml
global:
scrape_interval: 15s
scrape_configs:
- job_name: 'tomcat'
static_configs:
- targets: ['localhost:8080']
metrics_path: '/actuator/prometheus'
scrape_interval: 5s
监控指标配置
java
// 监控指标配置
@Component
public class TomcatMetrics {
private final MeterRegistry meterRegistry;
public TomcatMetrics(MeterRegistry meterRegistry) {
this.meterRegistry = meterRegistry;
}
@EventListener
public void handleRequest(RequestHandledEvent event) {
// 记录请求指标
Timer.Sample sample = Timer.start(meterRegistry);
sample.stop(Timer.builder("tomcat.request.duration")
.tag("method", event.getMethod())
.tag("status", String.valueOf(event.getStatusCode()))
.register(meterRegistry));
// 记录请求计数
Counter.builder("tomcat.request.count")
.tag("method", event.getMethod())
.tag("status", String.valueOf(event.getStatusCode()))
.register(meterRegistry)
.increment();
}
}
8.3 JMX 监控实现
8.3.1 JMX 配置
启用 JMX 监控
xml
<!-- JMX 配置 -->
<Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
<Listener className="org.apache.catalina.mbeans.JmxRemoteLifecycleListener"
rmiRegistryPortPlatform="10099"
rmiServerPortPlatform="10000" />
JMX 参数配置
bash
# JMX 参数配置
export JAVA_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=10099 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false"
8.3.2 JMX 监控实现
线程池监控
java
// 线程池监控
public class ThreadPoolMonitor {
private final MBeanServer mBeanServer;
public ThreadPoolMonitor() {
this.mBeanServer = ManagementFactory.getPlatformMBeanServer();
}
public void monitorThreadPools() {
try {
ObjectName threadPoolName = new ObjectName("Catalina:type=ThreadPool,name=*");
Set<ObjectName> threadPools = mBeanServer.queryNames(threadPoolName, null);
for (ObjectName threadPool : threadPools) {
int currentThreadCount = (Integer) mBeanServer.getAttribute(threadPool, "currentThreadCount");
int currentThreadsBusy = (Integer) mBeanServer.getAttribute(threadPool, "currentThreadsBusy");
int maxThreads = (Integer) mBeanServer.getAttribute(threadPool, "maxThreads");
int minSpareThreads = (Integer) mBeanServer.getAttribute(threadPool, "minSpareThreads");
int maxSpareThreads = (Integer) mBeanServer.getAttribute(threadPool, "maxSpareThreads");
System.out.println("Thread Pool: " + threadPool.getKeyProperty("name"));
System.out.println("Current Threads: " + currentThreadCount);
System.out.println("Busy Threads: " + currentThreadsBusy);
System.out.println("Max Threads: " + maxThreads);
System.out.println("Min Spare Threads: " + minSpareThreads);
System.out.println("Max Spare Threads: " + maxSpareThreads);
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
内存监控
java
// 内存监控
public class MemoryMonitor {
private final MBeanServer mBeanServer;
public MemoryMonitor() {
this.mBeanServer = ManagementFactory.getPlatformMBeanServer();
}
public void monitorMemory() {
try {
ObjectName memoryName = new ObjectName("java.lang:type=Memory");
CompositeData heapUsage = (CompositeData) mBeanServer.getAttribute(memoryName, "HeapMemoryUsage");
CompositeData nonHeapUsage = (CompositeData) mBeanServer.getAttribute(memoryName, "NonHeapMemoryUsage");
long heapUsed = (Long) heapUsage.get("used");
long heapMax = (Long) heapUsage.get("max");
long nonHeapUsed = (Long) nonHeapUsage.get("used");
long nonHeapMax = (Long) nonHeapUsage.get("max");
System.out.println("Heap Memory: " + heapUsed / 1024 / 1024 + "MB / " + heapMax / 1024 / 1024 + "MB");
System.out.println("Non-Heap Memory: " + nonHeapUsed / 1024 / 1024 + "MB / " + nonHeapMax / 1024 / 1024 + "MB");
} catch (Exception e) {
e.printStackTrace();
}
}
}
8.3.3 自定义 MBean
自定义 MBean 接口
java
// 自定义 MBean 接口
public interface TomcatMonitorMBean {
int getActiveConnections();
int getTotalRequests();
double getAverageResponseTime();
void resetCounters();
}
自定义 MBean 实现
java
// 自定义 MBean 实现
public class TomcatMonitor implements TomcatMonitorMBean {
private final AtomicInteger activeConnections = new AtomicInteger(0);
private final AtomicLong totalRequests = new AtomicLong(0);
private final AtomicLong totalResponseTime = new AtomicLong(0);
@Override
public int getActiveConnections() {
return activeConnections.get();
}
@Override
public int getTotalRequests() {
return totalRequests.intValue();
}
@Override
public double getAverageResponseTime() {
long requests = totalRequests.get();
if (requests == 0) {
return 0.0;
}
return (double) totalResponseTime.get() / requests;
}
@Override
public void resetCounters() {
totalRequests.set(0);
totalResponseTime.set(0);
}
public void recordRequest(long responseTime) {
totalRequests.incrementAndGet();
totalResponseTime.addAndGet(responseTime);
}
}
注册自定义 MBean
java
// 注册自定义 MBean
public class MBeanRegistration {
public void registerMBean() {
try {
MBeanServer mBeanServer = ManagementFactory.getPlatformMBeanServer();
ObjectName objectName = new ObjectName("com.example:type=TomcatMonitor");
TomcatMonitor monitor = new TomcatMonitor();
mBeanServer.registerMBean(monitor, objectName);
} catch (Exception e) {
e.printStackTrace();
}
}
}
8.4 性能监控与诊断
8.4.1 性能指标监控
关键性能指标
| 指标 | 说明 | 正常范围 | 告警阈值 |
|---|---|---|---|
| 响应时间 | 平均响应时间 | < 100ms | > 500ms |
| 吞吐量 | 每秒请求数 | > 1000 req/s | < 500 req/s |
| 并发连接数 | 当前连接数 | < 1000 | > 2000 |
| 线程池使用率 | 线程池使用率 | < 80% | > 90% |
| 内存使用率 | 堆内存使用率 | < 80% | > 90% |
| GC 频率 | GC 频率 | < 10次/分钟 | > 50次/分钟 |
性能监控实现
java
// 性能监控实现
@Component
public class PerformanceMonitor {
private final AtomicLong requestCount = new AtomicLong(0);
private final AtomicLong totalResponseTime = new AtomicLong(0);
private final AtomicLong maxResponseTime = new AtomicLong(0);
private final AtomicLong minResponseTime = new AtomicLong(Long.MAX_VALUE);
@EventListener
public void handleRequest(RequestHandledEvent event) {
long responseTime = event.getProcessingTimeMillis();
requestCount.incrementAndGet();
totalResponseTime.addAndGet(responseTime);
// 更新最大响应时间
long currentMax = maxResponseTime.get();
while (responseTime > currentMax && !maxResponseTime.compareAndSet(currentMax, responseTime)) {
currentMax = maxResponseTime.get();
}
// 更新最小响应时间
long currentMin = minResponseTime.get();
while (responseTime < currentMin && !minResponseTime.compareAndSet(currentMin, responseTime)) {
currentMin = minResponseTime.get();
}
}
public PerformanceStats getStats() {
long count = requestCount.get();
long total = totalResponseTime.get();
long max = maxResponseTime.get();
long min = minResponseTime.get();
return new PerformanceStats(count, total / count, max, min);
}
}
8.4.2 内存泄漏检测
内存泄漏检测
java
// 内存泄漏检测
public class MemoryLeakDetector {
private final MemoryMXBean memoryBean;
private final ScheduledExecutorService scheduler;
public MemoryLeakDetector() {
this.memoryBean = ManagementFactory.getMemoryMXBean();
this.scheduler = Executors.newScheduledThreadPool(1);
startMonitoring();
}
private void startMonitoring() {
scheduler.scheduleAtFixedRate(() -> {
MemoryUsage heapUsage = memoryBean.getHeapMemoryUsage();
long used = heapUsage.getUsed();
long max = heapUsage.getMax();
double usagePercent = (double) used / max * 100;
if (usagePercent > 80) {
System.out.println("Warning: High memory usage: " + usagePercent + "%");
// 触发内存分析
analyzeMemoryUsage();
}
}, 0, 30, TimeUnit.SECONDS);
}
private void analyzeMemoryUsage() {
// 分析内存使用情况
MemoryMXBean memoryBean = ManagementFactory.getMemoryMXBean();
MemoryUsage heapUsage = memoryBean.getHeapMemoryUsage();
System.out.println("Memory Analysis:");
System.out.println("Used: " + heapUsage.getUsed() / 1024 / 1024 + "MB");
System.out.println("Max: " + heapUsage.getMax() / 1024 / 1024 + "MB");
System.out.println("Usage: " + (double) heapUsage.getUsed() / heapUsage.getMax() * 100 + "%");
}
}
8.4.3 线程死锁检测
线程死锁检测
java
// 线程死锁检测
public class DeadlockDetector {
private final ThreadMXBean threadBean;
private final ScheduledExecutorService scheduler;
public DeadlockDetector() {
this.threadBean = ManagementFactory.getThreadMXBean();
this.scheduler = Executors.newScheduledThreadPool(1);
startMonitoring();
}
private void startMonitoring() {
scheduler.scheduleAtFixedRate(() -> {
long[] deadlockedThreads = threadBean.findDeadlockedThreads();
if (deadlockedThreads != null && deadlockedThreads.length > 0) {
System.out.println("Deadlock detected in threads: " + Arrays.toString(deadlockedThreads));
// 获取死锁线程信息
ThreadInfo[] threadInfos = threadBean.getThreadInfo(deadlockedThreads);
for (ThreadInfo threadInfo : threadInfos) {
System.out.println("Deadlocked thread: " + threadInfo.getThreadName());
System.out.println("Thread state: " + threadInfo.getThreadState());
System.out.println("Lock info: " + threadInfo.getLockInfo());
}
}
}, 0, 10, TimeUnit.SECONDS);
}
}
8.5 故障排查与调试
8.5.1 常见故障类型
故障分类
| 故障类型 | 症状 | 原因 | 解决方案 |
|---|---|---|---|
| 内存溢出 | OutOfMemoryError | 内存不足 | 增加堆内存,优化代码 |
| 线程池耗尽 | 请求超时 | 线程池配置不当 | 调整线程池参数 |
| 连接超时 | 连接失败 | 网络问题 | 检查网络配置 |
| 应用启动失败 | 启动异常 | 配置错误 | 检查配置文件 |
| 性能下降 | 响应慢 | 资源不足 | 优化配置,增加资源 |
8.5.2 故障诊断工具
线程转储分析
bash
# 生成线程转储
jstack <pid> > thread_dump.txt
# 分析线程转储
grep "BLOCKED" thread_dump.txt
grep "WAITING" thread_dump.txt
grep "RUNNABLE" thread_dump.txt
堆转储分析
bash
# 生成堆转储
jmap -dump:format=b,file=heap_dump.hprof <pid>
# 分析堆转储
jhat heap_dump.hprof
8.5.3 调试技巧
远程调试配置
bash
# 远程调试参数
export JAVA_OPTS="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5005"
日志级别调整
properties
# 调试日志配置
org.apache.catalina.level = DEBUG
org.apache.coyote.level = DEBUG
org.apache.tomcat.level = DEBUG
8.6 自动化监控方案
8.6.1 监控脚本
健康检查脚本
bash
#!/bin/bash
# health_check.sh
# 检查 Tomcat 进程
if ! pgrep -f "tomcat" > /dev/null; then
echo "Tomcat process not found"
exit 1
fi
# 检查端口
if ! netstat -ln | grep :8080 > /dev/null; then
echo "Port 8080 not listening"
exit 1
fi
# 检查 HTTP 响应
if ! curl -f http://localhost:8080/ > /dev/null 2>&1; then
echo "HTTP response failed"
exit 1
fi
echo "Tomcat is healthy"
exit 0
性能监控脚本
bash
#!/bin/bash
# performance_monitor.sh
# 获取性能指标
THREAD_COUNT=$(ps -eLf | grep tomcat | wc -l)
MEMORY_USAGE=$(ps -o pid,vsz,rss,comm -p $(pgrep -f tomcat) | tail -1 | awk '{print $3}')
CPU_USAGE=$(ps -o pid,pcpu,comm -p $(pgrep -f tomcat) | tail -1 | awk '{print $2}')
echo "Thread Count: $THREAD_COUNT"
echo "Memory Usage: $MEMORY_USAGE KB"
echo "CPU Usage: $CPU_USAGE%"
# 检查阈值
if [ $THREAD_COUNT -gt 1000 ]; then
echo "Warning: High thread count"
fi
if [ $MEMORY_USAGE -gt 1000000 ]; then
echo "Warning: High memory usage"
fi
if [ $(echo "$CPU_USAGE > 80" | bc) -eq 1 ]; then
echo "Warning: High CPU usage"
fi
8.6.2 告警机制
邮件告警
java
// 邮件告警
@Component
public class AlertService {
@Autowired
private JavaMailSender mailSender;
public void sendAlert(String subject, String message) {
try {
MimeMessage mimeMessage = mailSender.createMimeMessage();
MimeMessageHelper helper = new MimeMessageHelper(mimeMessage, true);
helper.setTo("admin@example.com");
helper.setSubject(subject);
helper.setText(message);
mailSender.send(mimeMessage);
} catch (Exception e) {
e.printStackTrace();
}
}
}
短信告警
java
// 短信告警
@Component
public class SmsAlertService {
public void sendSmsAlert(String message) {
// 发送短信告警
System.out.println("SMS Alert: " + message);
}
}
8.6.3 监控面板
监控面板实现
java
// 监控面板
@RestController
@RequestMapping("/monitor")
public class MonitorController {
@Autowired
private PerformanceMonitor performanceMonitor;
@GetMapping("/stats")
public PerformanceStats getStats() {
return performanceMonitor.getStats();
}
@GetMapping("/health")
public Map<String, Object> getHealth() {
Map<String, Object> health = new HashMap<>();
health.put("status", "UP");
health.put("timestamp", System.currentTimeMillis());
return health;
}
}
8.7 本章小结
关键要点
-
日志系统:
- 了解各种日志文件的作用
- 配置合适的日志级别
- 自定义日志格式
-
监控方式:
- 使用内置监控工具
- 集成外部监控工具
- 实现自定义监控
-
JMX 监控:
- 配置 JMX 监控
- 实现自定义 MBean
- 监控关键指标
-
性能监控:
- 监控关键性能指标
- 检测内存泄漏
- 检测线程死锁
-
故障排查:
- 识别常见故障类型
- 使用诊断工具
- 应用调试技巧
-
自动化监控:
- 实现监控脚本
- 建立告警机制
- 创建监控面板
最佳实践
-
监控策略:
- 建立全面的监控体系
- 设置合理的告警阈值
- 定期分析监控数据
-
故障处理:
- 建立故障处理流程
- 准备应急处理方案
- 记录故障处理经验
-
性能优化:
- 持续监控性能指标
- 及时调整配置参数
- 优化应用代码
下一步学习
在下一章中,我们将深入探讨 Tomcat 的源码阅读与扩展,了解 Tomcat 的核心实现原理,以及如何通过自定义组件扩展 Tomcat 的功能。
相关资源: