一、问题描述
项目部署在 CentOS 服务器上。项目偶尔会出现无响应的情况,这时理所当然要上去用 JDK 相关命令看看堆栈和GC等信息了。
进入 Java 程序所在容器:docekr-compose exec api bash,进入到 api 容器的 bash 终端。
jps 打印 Java 进程:
# jps
11 wallet-coin-1.0-SNAPSHOT.jar
1758 Jps
嗯,jps 命令还是能正常使用的,api.jar 程序的进程号是1。
jmap 命令打印堆栈摘要信息:jmap -heap 11,然而,报错了!
bash-4.4# jmap -heap 11
Attaching to process ID 11, please wait...
Error attaching to process: sun.jvm.hotspot.debugger.DebuggerException: Can't attach to the process: ptrace(PTRACE_ATTACH, ..) failed for 11: Operation not permitted
sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException: Can't attach to the process: ptrace(PTRACE_ATTACH, ..) failed for 11: Operation not permitted
at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:163)
at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.attach(LinuxDebuggerLocal.java:278)
at sun.jvm.hotspot.HotSpotAgent.attachDebugger(HotSpotAgent.java:671)
at sun.jvm.hotspot.HotSpotAgent.setupDebuggerLinux(HotSpotAgent.java:611)
at sun.jvm.hotspot.HotSpotAgent.setupDebugger(HotSpotAgent.java:337)
at sun.jvm.hotspot.HotSpotAgent.go(HotSpotAgent.java:304)
at sun.jvm.hotspot.HotSpotAgent.attach(HotSpotAgent.java:140)
at sun.jvm.hotspot.tools.Tool.start(Tool.java:185)
at sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
at sun.jvm.hotspot.tools.HeapSummary.main(HeapSummary.java:49)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.tools.jmap.JMap.runTool(JMap.java:201)
at sun.tools.jmap.JMap.main(JMap.java:130)
Caused by: sun.jvm.hotspot.debugger.DebuggerException: Can't attach to the process: ptrace(PTRACE_ATTACH, ..) failed for 11: Operation not permitted
at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.attach0(Native Method)
at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.access$100(LinuxDebuggerLocal.java:62)
at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1AttachTask.doit(LinuxDebuggerLocal.java:269)
at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:138)
二、解决方案
这其实不是什么 Bug,而是 Docker 自 1.10 版本开始加入的安全特性。
类似于 jmap 这些 JDK 工具依赖于 Linux 的 PTRACE_ATTACH,而是 Docker 自 1.10 在默认的 seccomp 配置文件中禁用了 ptrace。
这篇文章介绍了整个的缘由以及应对方法:JVM in Docker and PTRACE_ATTACH
主要提及三种:
2.1 --security-opt seccomp=unconfined
简单暴力(不推荐),直接关闭 seccomp 配置。用法:
docker run --security-opt seccomp:unconfined ...
2.2 --cap-add=SYS_PTRACE
使用 --cap-add 明确添加指定功能:
docker run --cap-add=SYS_PTRACE ...
2.3 Docker Compose 的支持
Docker Compose 自 version 1.1.0 (2015-02-25) 起支持 cap_add。官方文档:cap_add, cap_drop。用法:
docker-compose.yml 改写后文件内容如下(相同内容部分就不重复贴了):
version: '2'
services:
mysql:
...
api:
...
cap_add:
- SYS_PTRACE
三、验证
因为我是单个容器,所以选择了第二种解决方法,将docker容器关闭删除后,在docker容器启动命令中加入:--cap-add=SYS_PTRACE,容器启动执行jmap -heap 11可以打印服务gc信息了
# jmap -heap 11
Attaching to process ID 11, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.172-b11
using parallel threads in the new generation.
using thread-local object allocation.
Concurrent Mark-Sweep GC
Heap Configuration:
MinHeapFreeRatio = 40
MaxHeapFreeRatio = 70
MaxHeapSize = 1073741824 (1024.0MB)
NewSize = 134217728 (128.0MB)
MaxNewSize = 134217728 (128.0MB)
OldSize = 939524096 (896.0MB)
NewRatio = 2
SurvivorRatio = 8
MetaspaceSize = 134217728 (128.0MB)
CompressedClassSpaceSize = 125829120 (120.0MB)
MaxMetaspaceSize = 134217728 (128.0MB)
G1HeapRegionSize = 0 (0.0MB)
Heap Usage:
New Generation (Eden + 1 Survivor Space):
capacity = 120848384 (115.25MB)
used = 61863776 (58.997894287109375MB)
free = 58984608 (56.252105712890625MB)
51.19123148556128% used
Eden Space:
capacity = 107479040 (102.5MB)
used = 57787616 (55.110565185546875MB)
free = 49691424 (47.389434814453125MB)
53.766405059070124% used
From Space:
capacity = 13369344 (12.75MB)
used = 4076160 (3.8873291015625MB)
free = 9293184 (8.8626708984375MB)
30.488855698529413% used
To Space:
capacity = 13369344 (12.75MB)
used = 0 (0.0MB)
free = 13369344 (12.75MB)
0.0% used
concurrent mark-sweep generation:
capacity = 939524096 (896.0MB)
used = 93429680 (89.10148620605469MB)
free = 846094416 (806.8985137939453MB)
9.94436229978289% used
39322 interned Strings occupying 4867000 bytes.