概述
在ubuntu环境下,使用使用breakpad生成和解析minidump来分析C++程序问题。记录下来作为笔记备用。
步骤
1. 下载breakpad
git clone https://github.com/google/breakpad.git
2. 编译
命令行进入breakpad文件夹下:
./configure
make
sudo make install
可能遇到的问题:
推荐使用g++-13来作为g++的版本。
问题1:g++需要支持C++20,电脑原来是g++-9报错,后来升级到g++-13解决。
问题2:make时报错:
cpp
In file included from ./src/client/linux/dump_writer_common/thread_info.h:36,
from ./src/client/linux/minidump_writer/linux_dumper.h:53,
from ./src/client/linux/minidump_writer/minidump_writer.h:41,
from src/tools/linux/core2md/core2md.cc:37:
./src/common/memory_allocator.h:51:10: fatal error: third_party/lss/linux_syscall_support.h: 没有那个文件或目录
51 | #include "third_party/lss/linux_syscall_support.h"
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:5671:src/tools/linux/core2md/core2md.o] 错误 1
解决方法:
在breakpad/src/third_party/lss/目录下,上网下载文件linux_syscall_support.h放进去,一个可用的地址是:https://gitee.com/keeyou/lss/blob/develop/linux_syscall_support.h
示例程序
cpp
#include <iostream>
#include <breakpad/client/linux/handler/exception_handler.h>
bool DumpCallback(const google_breakpad::MinidumpDescriptor& descriptor, void* context, bool succeeded)
{
std::cout<<"Dump path:" << descriptor.path() << std::endl;
return succeeded;
}
int main()
{
google_breakpad::MinidumpDescriptor descriptor("./");
google_breakpad::ExceptionHandler eh(descriptor, nullptr, DumpCallback, nullptr, true, -1);
int *ptr = nullptr;
*ptr = 1; //这里有问题
return 0;
}
编译命令:
g++ -g minidump1.cpp -o minidump1 -I /usr/local/include/breakpad -lbreakpad_client -pthread
-I /usr/local/include/breakpad指定头文件的路径,防止找不到头文件
-g选项表示增加调试信息
-pthread是因为breakpad内部用到
-lbreakpad_client表示引用这个库
运行:
./minidump1
程序报错并生成.dmp文件
3.查看minidump文件信息
3.1,生成符号文件:(dump_syms programName > programName.sym)
dump_syms minidump1 > minidump1.sym
注意,dump_syms命令如果找不到,去下载的breakpad/src/tools/linux/dump_syms目录下找
minidump1.sym这个名字不是随便取的,需要和可执行文件同名
3.2. 设置符号文件的目录:
选一个位置,创建目录文件symbols,在它下面创建文件夹minidump1,注意,这个名字和可执行文件同名,查看上一步文件minidump1.sym的内容,从它的第一行获取UUID
例如:第一行:MODULE Linux x86_64 9F290E00150C9597A510DF9C550D7BCF0 minidump1
得到9F290E00150C9597A510DF9C550D7BCF0
在minidump1目录下创建目录9F290E00150C9597A510DF9C550D7BCF0,并把minidump1.sym放进去
最终目录为:
symbols/minidump1/9F290E00150C9597A510DF9C550D7BCF0/minidump1.sym
3.3. 使用minidump_stackwalk解析文件
如果找不到minidump_stackwalk,去breakpad目录下src/processor文件夹下找。
执行命令: minidump_stackwalk <UUID>.dmp symbols目录 > 栈信息.txt
例如:minidump_stackwalk 0f49290e-008a-42be-b3d19c93-3cbe7f82.dmp symbols/ > symbols/minidump1/9F290E00150C9597A510DF9C550D7BCF0/stack1.txt
前面的dmp文件和symbols文件都用上了
3.4. 查看文件stack1.txt
cpp
Operating system: Linux
0.0.0 Linux 6.8.0-101-generic #101~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Feb 11 13:19:54 UTC x86_64
CPU: amd64
family 25 model 33 stepping 0
4 CPUs
GPU: UNKNOWN
Crash reason: SIGSEGV /SEGV_MAPERR
Crash address: 0x0
Process uptime: not available
Thread 0 (crashed)
0 minidump1!main [minidump1.cpp : 16 + 0x7]
rax = 0x0000000000000000 rdx = 0x0000000000000000
rcx = 0x00005865141a8370 rbx = 0x0000000000000000
rsi = 0x0000000000000000 rdi = 0x00005865077b97e0
rbp = 0x00007fff5464df60 rsp = 0x00007fff5464ddb0
r8 = 0x0000000000000000 r9 = 0x00005865141a8370
r10 = 0x0000000000000008 r11 = 0x00007f8ebfc1ace0
r12 = 0x00007fff5464e078 r13 = 0x000058650779f2e2
r14 = 0x00005865077b89f8 r15 = 0x00007f8ec015c040
rip = 0x000058650779f3b6
Found by: given as instruction pointer in context
1 libc.so.6 + 0x29d90
rbx = 0x0000000000000000 rbp = 0x0000000000000001
rsp = 0x00007fff5464df70 r12 = 0x00007fff5464e078
r13 = 0x000058650779f2e2 r14 = 0x00005865077b89f8
r15 = 0x00007f8ec015c040 rip = 0x00007f8ebfa29d90
Found by: call frame info
2 minidump1!DumpCallback(google_breakpad::MinidumpDescriptor const&, void*, bool) [minidump1.cpp : 7 + 0x6]
rsp = 0x00007fff5464df80 rip = 0x000058650779f2e2
Found by: stack scanning
3 0x100000000
rbx = 0x0000000000000000 rbp = 0x000058650779f2e2
rsp = 0x00007fff5464df88 rip = 0x0000000100000000
Found by: call frame info
4 minidump1!DumpCallback(google_breakpad::MinidumpDescriptor const&, void*, bool) [minidump1.cpp : 7 + 0x6]
rsp = 0x00007fff5464dfb0 rip = 0x000058650779f2e2
Found by: stack scanning
5 0x5865077b89f8
rbx = 0x00007fff5464e078 rbp = 0x000058650779f2e2
rsp = 0x00007fff5464dfb8 rip = 0x00005865077b89f8
Found by: call frame info
6 libc.so.6 + 0x29e40
rsp = 0x00007fff5464e010 rip = 0x00007f8ebfa29e40
Found by: stack scanning
7 minidump1!google_breakpad::ConvertUTF8toUTF32(unsigned char const**, unsigned char const*, unsigned long**, unsigned long*, google_breakpad::ConversionFlags) [clone .cold] + 0xe
rsp = 0x00007fff5464e040 rip = 0x000058650779f190
Found by: stack scanning
8 minidump1!_start + 0x25
rsp = 0x00007fff5464e060 rip = 0x000058650779f1b5
Found by: stack scanning
9 0x7fff5464e068
rsp = 0x00007fff5464e068 rip = 0x00007fff5464e068
Found by: call frame info
Loaded modules:
0x58650779b000 - 0x5865077b1fff minidump1 ??? (main)
0x7f8ebfa00000 - 0x7f8ebfbbcfff libc.so.6 ??? (WARNING: No symbols, libc.so.6, A17B5C09AE4881CA668091F7180470780)
0x7f8ebfd19000 - 0x7f8ebfda2fff libm.so.6 ???
0x7f8ebfe00000 - 0x7f8ebffe9fff libstdc++.so.6 ???
0x7f8ec00e1000 - 0x7f8ec0107fff libgcc_s.so.1 ???
0x7f8ec0122000 - 0x7f8ec014dfff ld-linux-x86-64.so.2 ???
0x7fff54694000 - 0x7fff54695fff linux-gate.so ???
可以看出,minidump1!main [minidump1.cpp : 16 + 0x7] 这里指出了问题位置在16行。
注意要严格遵守上面的步骤,如果不正确创建symbols目录或者sym文件名字不对,则最后解析的文件无法指明错误位置,知识一堆地址。
进一步探索
如果解析文件失败了,没有显示正确位置,用以下方法实现定位:
1.定位地址
错误复现:
比如上面步骤3写成了 minidump_stackwalk 0f49290e-008a-42be-b3d19c93-3cbe7f82.dmp symbols/minidump1/ > symbols/minidump1/9F290E00150C9597A510DF9C550D7BCF0/stack2.txt
查看 stack2.txt文件
cpp
Operating system: Linux
0.0.0 Linux 6.8.0-101-generic #101~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Feb 11 13:19:54 UTC x86_64
CPU: amd64
family 25 model 33 stepping 0
4 CPUs
GPU: UNKNOWN
Crash reason: SIGSEGV /SEGV_MAPERR
Crash address: 0x0
Process uptime: not available
Thread 0 (crashed)
0 minidump1 + 0x43b6
rax = 0x0000000000000000 rdx = 0x0000000000000000
rcx = 0x00005865141a8370 rbx = 0x0000000000000000
rsi = 0x0000000000000000 rdi = 0x00005865077b97e0
rbp = 0x00007fff5464df60 rsp = 0x00007fff5464ddb0
r8 = 0x0000000000000000 r9 = 0x00005865141a8370
r10 = 0x0000000000000008 r11 = 0x00007f8ebfc1ace0
r12 = 0x00007fff5464e078 r13 = 0x000058650779f2e2
r14 = 0x00005865077b89f8 r15 = 0x00007f8ec015c040
rip = 0x000058650779f3b6
Found by: given as instruction pointer in context
1 minidump1 + 0x4279
rbp = 0x00007fff5464df60 rsp = 0x00007fff5464de70
rip = 0x000058650779f279
Found by: stack scanning
2 libc.so.6 + 0x29d90
rsp = 0x00007fff5464df70 rip = 0x00007f8ebfa29d90
Found by: stack scanning
3 minidump1 + 0x42e2
rsp = 0x00007fff5464df80 rip = 0x000058650779f2e2
Found by: stack scanning
4 minidump1 + 0x42e2
rsp = 0x00007fff5464dfb0 rip = 0x000058650779f2e2
Found by: stack scanning
5 libc.so.6 + 0x29e40
rsp = 0x00007fff5464e010 rip = 0x00007f8ebfa29e40
Found by: stack scanning
6 minidump1 + 0x4190
rsp = 0x00007fff5464e040 rip = 0x000058650779f190
Found by: stack scanning
7 minidump1 + 0x41b5
rsp = 0x00007fff5464e060 rip = 0x000058650779f1b5
Found by: stack scanning
Loaded modules:
0x58650779b000 - 0x5865077b1fff minidump1 ??? (main) (WARNING: No symbols, minidump1, 9F290E00150C9597A510DF9C550D7BCF0)
0x7f8ebfa00000 - 0x7f8ebfbbcfff libc.so.6 ??? (WARNING: No symbols, libc.so.6, A17B5C09AE4881CA668091F7180470780)
0x7f8ebfd19000 - 0x7f8ebfda2fff libm.so.6 ???
0x7f8ebfe00000 - 0x7f8ebffe9fff libstdc++.so.6 ???
0x7f8ec00e1000 - 0x7f8ec0107fff libgcc_s.so.1 ???
0x7f8ec0122000 - 0x7f8ec014dfff ld-linux-x86-64.so.2 ???
0x7fff54694000 - 0x7fff54695fff linux-gate.so ???
上面内容可以看出, 0 minidump1 + 0x43b6是停止位置。记录地址是0x43b6
2,解析地址位置
objdump -S minidump1 > objdump1_S.txt
这句指令会把 minidump1 的二进制文件的反汇编代码 + 源代码(如果有调试信息) 导出到 objdump1_S.txt 文本文件中,方便查看和分析。
打开objdump1_S.txt文件,查找43b6,会发现
int *ptr = nullptr;
43a4: 48 c7 85 60 fe ff ff movq $0x0,-0x1a0(%rbp)
43ab: 00 00 00 00
*ptr = 1; //这里有问题
43af: 48 8b 85 60 fe ff ff mov -0x1a0(%rbp),%rax
43b6: c7 00 01 00 00 00 movl $0x1,(%rax)
return 0;

43b6所对应的源码是*ptr = 1;定位出问题位置