Cross Debugging with GDB: Embedded Linux

This is suitable for environments with limited resources and insufficient memory, to avoid out-of-memory errors caused by running gdb.

The uclibc platform does not yet support this functionality.

文章目录

- [Install gdb-multiarch on PC](#Install gdb-multiarch on PC)
- [Run gdbserver on DUT](#Run gdbserver on DUT)
- [Prepare sysroot](#Prepare sysroot)
- [Run gdb-multiiarch on PC](#Run gdb-multiiarch on PC)
- [GDB Debug by Reading Coredump](#GDB Debug by Reading Coredump)
- - [About coredump](#About coredump)
  - [Prepare FW with coredump enabled](#Prepare FW with coredump enabled)
  - [Enable coredump size & name](#Enable coredump size & name)
  - [Permernently enable coredump size when build SDK](#Permernently enable coredump size when build SDK)
- [Check Symbol From File](#Check Symbol From File)
- - [Check library or program dependencies](#Check library or program dependencies)
  - [Make sure that the debugging program have symbols](#Make sure that the debugging program have symbols)

Install gdb-multiarch on PC

install MINGW64 MSYS2 environment

in MSYS2 environment install MINGW gdb-multiarch

bash 复制代码

$ pacman -S mingw-w64-x86_64-gdb-multiarch

press 'Y' to install all depended libraries

Run gdbserver on DUT

prepare gdbserver on buildroot for target platform
change gdbserver $ chmod 755 gdbserver
gdbserver cmd
./gdbserver localhost:2001 <bin + args>
check LD_LIBRARY_PATH

bash 复制代码

$ echo $LD_LIBRARY_PATH
/system/lib

Prepare sysroot

Downlaod target_debug to C:\msys64\home\imxx\ 下, this is MYSYS2 home

Run gdb-multiiarch on PC

open MINGW

bash 复制代码

(gdb) gdb-multiarch ./testOnDemandRTSPServer # 這個檔案必須是DUT 上執行的相同一個
//(gdb) gdb-multiarch ./target_debug/system/bin/testOnDemandRTSPServer   # 或是直接執行 sysroot 裡面的對應執行檔
(gdb) set architecture arm
(gdb) set sysroot ./target_debug # 這就是方才複製 SDK/fs/rootfs/output/target_debug 位置
(gdb) set solib-search-path ./target_debug/system/lib # 此位置必須取代 DUT $LD_LIBRARY_PATH
(gdb) target remote 172.16.35.42:2001 # 也就是 DUT IP + gdb server port
(gdb) b main # 設置中斷點在 main function
(gdb) continue

Check shared library info

bash 复制代码

(gdb)  info shared

output

bash 复制代码

(gdb) info shared
From        To          Syms Read   Shared Object Library
0xb6f84800  0xb6f97408  Yes         ./target_debug/lib/ld-linux-armhf.so.3
0xb6f3c930  0xb6f6c860  Yes         ./target_debug/system/lib/libinf.so.1
0xb6f235e8  0xb6f23cd4  Yes         ./target_debug/system/lib/libaftr.so.3
0xb6f082e8  0xb6f0fc28  Yes         ./target_debug/system/lib/libvftr.so.3
0xb6d86704  0xb6e7c618  Yes         ./target_debug/system/lib/libmpp.so.3
0xb6daee78  0xb6db1d8c  Yes         ./target_debug/usr/lib/libjson-c.so.2
0xb6d98810  0xb6d9ae68  Yes         ./target_debug/lib/librt.so.1
0xb6d1c9b8  0xb6d6f76c  Yes         ./target_debug/usr/lib/libasound.so.2
0xb6c07d58  0xb6cd81fc  Yes         ./target_debug/usr/lib/libtensorflow-lite-optim.so
0xb6bb4974  0xb6bbbd4c  Yes         ./target_debug/usr/lib/libz.so.1
0xb6b6b0d0  0xb6b99c70  Yes         ./target_debug/usr/lib/libcurl.so.4
0xb6b1a414  0xb6b44020  Yes         ./target_debug/usr/lib/libssl.so.1.1
0xb69bd000  0xb6a9c950  Yes         ./target_debug/usr/lib/libcrypto.so.1.1
0xb6906048  0xb694d398  Yes         ./target_debug/lib/libstdc++.so.6
                        No          /lib/libm.so.6
0xb682fdf8  0xb683abe4  Yes         ./target_debug/lib/libgcc_s.so.1
                        No          /lib/libpthread.so.0
                        No          /lib/libc.so.6
0xb66fb97c  0xb66fc234  Yes         ./target_debug/lib/libdl.so.2
                        No          /lib/libatomic.so.1
                        No          /lib/libnss_files.so.2

GDB Debug by Reading Coredump

About coredump

A core dump, also known as a "core dump," is a snapshot of memory taken at the moment a process crashes abruptly during its execution.

When a program encounters an unhandled exception internally within a process, the operating system saves a dump of the process's current memory, register states, running stack, and other information into a file. This file is a binary file that can be analyzed using tools like gdb, elfdump, and objdump to examine specific contents.

Reading a core dump typically consumes less memory compared to running the binary with libraries containing symbols, as it doesn't require loading those libraries during the analysis process.

Prepare FW with coredump enabled

Core dumps are not enabled by default in the most SDK. You need to recompile the Linux kernel configuration to generate the firmware.

bash 复制代码

$ cd SDK/linux; make menuconfig
General setup -> configure standard kernel features -> Enable ELF core dumps

Enable coredump size & name

file size

bash 复制代码

$ ulimit -a 
core file size (blocks)         (-c) 0 << 
data seg size (kb)              (-d) unlimited
scheduling priority             (-e) 0
file size (blocks)              (-f) unlimited
pending signals                 (-i) 3321
max locked memory (kb)          (-l) 64
max memory size (kb)            (-m) unlimited
open files                      (-n) 1024
POSIX message queues (bytes)    (-q) 819200
real-time priority              (-r) 0
stack size (kb)                 (-s) 8192
cpu time (seconds)              (-t) unlimited
max user processes              (-u) 3321
virtual memory (kb)             (-v) unlimited
file locks                      (-x) unlimited
$  ulimit -c unlmited << set coredump unlimited size

coredump path

bash 复制代码

$ cat  /proc/sys/kernel/core_pattern
core
$ echo "/tmp/core.%p" > /proc/sys/kernel/core_pattern --> set to pid name

coredump naming rules

GDB read coredump

you need to prepare target binary with debug symbol & sysroot
active GDB

bash 复制代码

$ ./gdb <binary with debug symbol> <coredump name>
$ (gdb) set sysroot ./target_debug/
$ info shared  << 查看有關library
$ info stack << 查看發生segfault 的函數傳遞

Permernently enable coredump size when build SDK

ulimit needs to be used after the user logs in and applies only to that logged-in environment.

To change the default ulimit of Linux kernel so that it takes effect during the rcS stage, modifications are required in linux/include/asm-generic/resource.h. However, defining this could lead to customers altering our Linux default values, which is why SDK does not use it by default.

bash 复制代码

[RLIMIT_CORE]        = {   64*1024*1024,  RLIM_INFINITY }

Check Symbol From File

Check library or program dependencies

Before debugging, you may need to know the dependency of target program

bash 复制代码

$ arm-augentix-linux-gnueabi-ldd --root . <file>
        libpthread.so.0 => /lib/libpthread.so.0 (0x8badf00d)
        libc.so.6 => /lib/libc.so.6 (0x8badf00d)
        ld-linux.so.3 => /lib/ld-linux.so.3 (0x8badf00d)
(ignore)

Make sure that the debugging program have symbols

You might check whether debug symbols exist with utils file in DUT.

bash 复制代码

$ file ld-2.22.so
ld-2.22.so: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, not stripped

On the other hand, check whether the section debug_info is in the ELF works also

bash 复制代码

$ arm-augentix-linux-gnueabi-objdump --syms ld-2.22.so | grep .debug_info
00000000 l    d  .debug_info    00000000              .debug_info