聊聊jvm的mapped buffer的统计

序

本文主要研究一下jvm的mapped buffer的统计

示例

private void writeDirectBuffer() {
    // 分配一个256MB的直接缓冲区
    ByteBuffer buffer = ByteBuffer.allocateDirect(256 * 1024 * 1024);

    // 填充数据
    Random random = new Random();
    while (buffer.remaining() >= 4) {
        buffer.putInt(random.nextInt());
    }

    System.out.println("Allocated direct buffer with capacity " + buffer.capacity());
}

private void writeMappedBuffer() throws IOException {
    RandomAccessFile file = new RandomAccessFile("/tmp/test.txt", "rw");
    FileChannel channel = file.getChannel();
    MappedByteBuffer buffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, 128 * 1024 * 1024); // 映射128MB的空间
    // 随机写入数据
    for (int i = 0; i < buffer.capacity(); i++) {
        buffer.put((byte) i);
    }
}

jvm.buffer.memory.used

http://localhost:8080/actuator/metrics/jvm.buffer.memory.used

{
    "name": "jvm.buffer.memory.used",
    "description": "An estimate of the memory that the Java virtual machine is using for this buffer pool",
    "baseUnit": "bytes",
    "measurements": [
        {
            "statistic": "VALUE",
            "value": 402685952
        }
    ],
    "availableTags": [
        {
            "tag": "id",
            "values": [
                "direct",
                "mapped"
            ]
        }
    ]
}

jvm.buffer.memory.used分了direct和mapped两大类，一共用了384MB

mapped

http://localhost:8080/actuator/metrics/jvm.buffer.memory.used?tag=id:mapped

{
    "name": "jvm.buffer.memory.used",
    "description": "An estimate of the memory that the Java virtual machine is using for this buffer pool",
    "baseUnit": "bytes",
    "measurements": [
        {
            "statistic": "VALUE",
            "value": 134217728
        }
    ],
    "availableTags": []
}

可以看到这里mapped用了128MB

direct

http://localhost:8080/actuator/metrics/jvm.buffer.memory.used?tag=id:direct

{
    "name": "jvm.buffer.memory.used",
    "description": "An estimate of the memory that the Java virtual machine is using for this buffer pool",
    "baseUnit": "bytes",
    "measurements": [
        {
            "statistic": "VALUE",
            "value": 268500992
        }
    ],
    "availableTags": []
}

可以看到这里direct用了256MB

Native Memory Tracking

开启

java -XX:+UnlockDiagnosticVMOptions -XX:NativeMemoryTracking=summary -jar target/app.jar

监控

jcmd 2315 VM.native_memory summary scale=MB
2315:

Native Memory Tracking:

Total: reserved=5882MB, committed=1100MB
-                 Java Heap (reserved=4096MB, committed=577MB)
                            (mmap: reserved=4096MB, committed=577MB)

-                     Class (reserved=1066MB, committed=46MB)
                            (classes #7035)
                            (malloc=10MB #10793)
                            (mmap: reserved=1056MB, committed=37MB)

-                    Thread (reserved=36MB, committed=36MB)
                            (thread #37)
                            (stack: reserved=36MB, committed=36MB)

-                      Code (reserved=246MB, committed=16MB)
                            (malloc=2MB #3971)
                            (mmap: reserved=244MB, committed=13MB)

-                        GC (reserved=160MB, committed=148MB)
                            (malloc=10MB #219)
                            (mmap: reserved=150MB, committed=138MB)

-                  Internal (reserved=266MB, committed=266MB)
                            (malloc=266MB #10067)

-                    Symbol (reserved=9MB, committed=9MB)
                            (malloc=8MB #74334)
                            (arena=2MB #1)

-    Native Memory Tracking (reserved=2MB, committed=2MB)
                            (tracking overhead=2MB)

NTM的Internal包含了direct buffer的统计，不包含mapped buffer的占用，而BufferPoolMXBean包含了direct和mapped。

另外通过调用System.loadLibrary()加载的共享库分配的内存NTM没办法追踪到，MXBean貌似也没办法统计到。

小结

• 聊聊jvm的direct buffer统计
• Memory Types in JVM
• Native Memory Tracking in JVM
• How to monitor JVM native memory usage by enabling native memory tracking
• Troubleshooting Problems With Native (Off-Heap) Memory in Java Applications
• Can a Java Application Use More Memory Than the Heap Size?
• java-non-heap-memory-analyzes