dpdk19.08编译问题解决方案

一、问题背景

1.1 环境信息

项目 版本
操作系统 Ubuntu 16.04 LTS
DPDK 版本 19.08.2
GCC 版本 5.4.0
Binutils 2.26.1
目标架构 x86_64

1.2 原始错误信息

使用 ./usertools/dpdk-setup.sh 编译 DPDK 时遇到以下错误:

复制代码
  CC portal/dpaa2_hw_dpio.o
  PMDINFO portal/dpaa2_hw_dpio.o.pmd.c
  CC portal/dpaa2_hw_dpio.o.pmd.o
  LD portal/dpaa2_hw_dpio.o
ld: i386:x86-64 architecture of input file `portal/dpaa2_hw_dpio.o.pmd.o' is incompatible with i386 output
ld: Relocatable linking with relocations from format elf64-x86-64 (portal/dpaa2_hw_dpio.o) to format elf32-i386 (portal/dpaa2_hw_dpio.o.o) is not supported
/home/jerry/Desktop/dpdk-stable-19.08.2/mk/internal/rte.compile-pre.mk:114: recipe for target 'portal/dpaa2_hw_dpio.o' failed
make[6]: *** [portal/dpaa2_hw_dpio.o] Error 1
...
RTE_TARGET exported as x86_64-native-linuxapp-gcc

错误关键信息

  • 输入文件架构:elf64-x86-64(64 位)
  • 输出格式:elf32-i386(32 位)
  • 问题文件:portal/dpaa2_hw_dpio.o(属于 FSLMC/DPAA2 驱动)
  • 目标架构设置:x86_64-native-linuxapp-gcc(看起来是正确的)

二、调试过程

步骤 1:尝试修改配置文件禁用 ARM 驱动

首先尝试修改 config/common_linuxapp

复制代码
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2/usertools# sed -i 's/CONFIG_RTE_LIBRTE_FSLMC_BUS=y/CONFIG_RTE_LIBRTE_FSLMC_BUS=n/' config/common_linuxapp
sed: can't read config/common_linuxapp: No such file or directory
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2/usertools# sed -i 's/CONFIG_RTE_LIBRTE_DPAA2_PMD=y/CONFIG_RTE_LIBRTE_DPAA2_PMD=n/' config/common_linuxapp
sed: can't read config/common_linuxapp: No such file or directory

问题 :配置文件路径不对,当前在 usertools 目录下,且 DPDK 19.08 使用的是 config/common_base


步骤 2:回到正确目录,修改正确的配置文件

复制代码
cd /home/jerry/Desktop/dpdk-stable-19.08.2
ls config/

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# cd /home/jerry/Desktop/dpdk-stable-19.08.2
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# ls config/
arm64-armada-linuxapp-gcc        defconfig_x86_64-native-linuxapp-clang
arm64-armada-linux-gcc           defconfig_x86_64-native-linuxapp-gcc
arm64-armv8a-linuxapp-clang      defconfig_x86_64-native-linuxapp-icc
arm64-armv8a-linuxapp-gcc        defconfig_x86_64-native-linux-clang
arm64-armv8a-linux-clang         defconfig_x86_64-native-linux-gcc
arm64-armv8a-linux-gcc           defconfig_x86_64-native-linux-icc
arm64-bluefield-linuxapp-gcc     defconfig_x86_x32-native-linuxapp-gcc
arm64-bluefield-linux-gcc        defconfig_x86_x32-native-linux-gcc
arm64-dpaa2-linuxapp-gcc         meson.build
arm64-dpaa2-linux-gcc            ppc_64-power8-linuxapp-gcc
arm64-dpaa-linuxapp-gcc          ppc_64-power8-linux-gcc
arm64-dpaa-linux-gcc             rte_config.h
arm64-octeontx2-linuxapp-gcc     rte_compatibility_defines.h
arm64-octeontx2-linux-gcc        rte_config.h.in
arm64-stingray-linuxapp-gcc      x86_64-native-bsdapp-clang
arm64-stingray-linux-gcc         x86_64-native-bsdapp-gcc
arm64-thunderx2-linuxapp-gcc     x86_64-native-freebsd-clang
arm64-thunderx2-linux-gcc        x86_64-native-freebsd-gcc
arm64-thunderx-linuxapp-gcc      x86_64-native-linuxapp-clang
arm64-thunderx-linux-gcc         x86_64-native-linuxapp-gcc
arm64-xgene1-linuxapp-gcc        x86_64-native-linuxapp-icc
arm64-xgene1-linux-gcc           x86_64-native-linux-clang
arm-armv7a-linuxapp-gcc          x86_64-native-linux-gcc
arm-armv7a-linux-gcc             x86_64-native-linux-icc
common_base                      x86_x32-native-linuxapp-gcc
common_linux                     x86_x32-native-linux-gcc
defconfig_i686-native-linuxapp-gcc
defconfig_i686-native-linuxapp-icc
defconfig_i686-native-linux-gcc
defconfig_i686-native-linux-icc

找到 common_base 文件,查看 DPAA/FSLMC 配置状态:

复制代码
grep -E "FSLMC|DPAA2" config/common_base | head -20

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep -E "FSLMC|DPAA2" config/common_base | head -20
# Compile NXP DPAA2 FSL-MC Bus
CONFIG_RTE_LIBRTE_FSLMC_BUS=n
# Compile Support Libraries for NXP DPAA2
CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL=n
CONFIG_RTE_LIBRTE_DPAA2_USE_PHYS_IOVA=y
# Compile burst-oriented NXP DPAA2 PMD driver
CONFIG_RTE_LIBRTE_DPAA2_PMD=n
CONFIG_RTE_LIBRTE_DPAA2_DEBUG_DRIVER=n
# Compile NXP DPAA2 crypto sec driver for CAAM HW
CONFIG_RTE_LIBRTE_PMD_DPAA2_SEC=n
# Compile PMD for NXP DPAA2 event device
CONFIG_RTE_LIBRTE_PMD_DPAA2_EVENTDEV=n
# Compile PMD for NXP DPAA2 CMDIF raw device
CONFIG_RTE_LIBRTE_PMD_DPAA2_CMDIF_RAWDEV=n
# Compile PMD for NXP DPAA2 QDMA raw device
CONFIG_RTE_LIBRTE_PMD_DPAA2_QDMA_RAWDEV=n

配置文件中已经是 =n,说明问题不在这里。


步骤 3:清理并重新编译

复制代码
# 清理
rm -rf build
rm -f .config

# 重新配置并编译
make config T=x86_64-native-linuxapp-gcc
make -j$(nproc)

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# rm -rf build
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# rm -f .config
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# make config T=x86_64-native-linuxapp-gcc
Configuration done using x86_64-native-linuxapp-gcc
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# make -j$(nproc)
...
  CC portal/dpaa2_hw_dpio.o
  PMDINFO portal/dpaa2_hw_dpio.o.pmd.c
  CC portal/dpaa2_hw_dpio.o.pmd.o
  LD portal/dpaa2_hw_dpio.o
ld: i386:x86-64 architecture of input file `portal/dpaa2_hw_dpio.o.pmd.o' is incompatible with i386 output
ld: Relocatable linking with relocations from format elf64-x86-64 (portal/dpaa2_hw_dpio.o) to format elf32-i386 (portal/dpaa2_hw_dpio.o.o) is not supported
/home/jerry/Desktop/dpdk-stable-19.08.2/mk/internal/rte.compile-pre.mk:114: recipe for target 'portal/dpaa2_hw_dpio.o' failed
make[4]: *** [portal/dpaa2_hw_dpio.o] Error 1
...
make: *** [all] Error 2

问题依旧 :即使 config/common_base 中已经设置为 =n,编译时仍然在尝试编译 FSLMC 驱动。


步骤 4:检查 build/.config 实际配置

关键发现:make config 生成的 build/.config 才是实际使用的配置文件。

复制代码
grep -E "FSLMC|DPAA" build/.config | grep -v "^#"

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep -E "FSLMC|DPAA" build/.config | grep -v "^#"
CONFIG_RTE_LIBRTE_COMMON_DPAAX=y
CONFIG_RTE_LIBRTE_DPAA_BUS=y
CONFIG_RTE_LIBRTE_DPAA_MEMPOOL=y
CONFIG_RTE_LIBRTE_DPAA_PMD=y
CONFIG_RTE_LIBRTE_DPAA_HWDEBUG=n
CONFIG_RTE_LIBRTE_FSLMC_BUS=y
CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL=y
CONFIG_RTE_LIBRTE_DPAA2_USE_PHYS_IOVA=y
CONFIG_RTE_LIBRTE_DPAA2_PMD=y
CONFIG_RTE_LIBRTE_DPAA2_DEBUG_DRIVER=n
CONFIG_RTE_LIBRTE_PMD_DPAA2_SEC=y
CONFIG_RTE_LIBRTE_PMD_DPAA_SEC=y
CONFIG_RTE_LIBRTE_DPAA_MAX_CRYPTODEV=4
CONFIG_RTE_LIBRTE_PMD_DPAA_EVENTDEV=y
CONFIG_RTE_LIBRTE_PMD_DPAA2_EVENTDEV=y
CONFIG_RTE_LIBRTE_PMD_DPAA2_CMDIF_RAWDEV=y
CONFIG_RTE_LIBRTE_PMD_DPAA2_QDMA_RAWDEV=y

关键发现build/.config 中大量的 ARM 驱动配置仍然是 =yconfig/common_base 的修改没有生效。


步骤 5:直接修改 build/.config

复制代码
sed -i 's/\(CONFIG_RTE_LIBRTE.*DPAA.*\)=y/\1=n/g' build/.config
sed -i 's/\(CONFIG_RTE_LIBRTE.*FSLMC.*\)=y/\1=n/g' build/.config

# 验证
grep -E "FSLMC|DPAA" build/.config | grep "=y"

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i 's/\(CONFIG_RTE_LIBRTE.*DPAA.*\)=y/\1=n/g' build/.config
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i 's/\(CONFIG_RTE_LIBRTE.*FSLMC.*\)=y/\1=n/g' build/.config
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep -E "FSLMC|DPAA" build/.config | grep "=y"
CONFIG_RTE_LIBRTE_DPAA_MEMPOOL=y
CONFIG_RTE_LIBRTE_DPAA_PMD=y
CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL=y
CONFIG_RTE_LIBRTE_DPAA2_USE_PHYS_IOVA=y
CONFIG_RTE_LIBRTE_DPAA2_PMD=y
CONFIG_RTE_LIBRTE_PMD_DPAA2_SEC=y
CONFIG_RTE_LIBRTE_PMD_DPAA_SEC=y
CONFIG_RTE_LIBRTE_PMD_DPAA_EVENTDEV=y
CONFIG_RTE_LIBRTE_PMD_DPAA2_EVENTDEV=y
CONFIG_RTE_LIBRTE_PMD_DPAA2_CMDIF_RAWDEV=y
CONFIG_RTE_LIBRTE_PMD_DPAA2_QDMA_RAWDEV=y

问题 :sed 正则没有完全匹配所有配置项(有些配置项名称不包含 LIBRTE)。


步骤 6:彻底禁用所有 ARM 相关配置

需要更彻底地修改:

复制代码
# 彻底删除 build 目录重新开始
rm -rf build

# 先修改 config/common_base
sed -i 's/\(CONFIG_RTE_LIBRTE.*DPAA.*\)=y/\1=n/g' config/common_base
sed -i 's/\(CONFIG_RTE_LIBRTE.*FSLMC.*\)=y/\1=n/g' config/common_base
sed -i 's/\(CONFIG_RTE_LIBRTE.*OCTEONTX.*\)=y/\1=n/g' config/common_base
sed -i 's/\(CONFIG_RTE_LIBRTE.*OTX2.*\)=y/\1=n/g' config/common_base
sed -i 's/\(CONFIG_RTE_LIBRTE.*THUNDERX.*\)=y/\1=n/g' config/common_base

# 验证
grep -E "DPAA|FSLMC|OCTEON|OTX2|THUNDER" config/common_base | grep "=y" || echo "已全部禁用"

# 重新配置
make config T=x86_64-native-linuxapp-gcc

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# rm -rf build
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i 's/\(CONFIG_RTE_LIBRTE.*DPAA.*\)=y/\1=n/g' config/common_base
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i 's/\(CONFIG_RTE_LIBRTE.*FSLMC.*\)=y/\1=n/g' config/common_base
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i 's/\(CONFIG_RTE_LIBRTE.*OCTEONTX.*\)=y/\1=n/g' config/common_base
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i 's/\(CONFIG_RTE_LIBRTE.*OTX2.*\)=y/\1=n/g' config/common_base
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i 's/\(CONFIG_RTE_LIBRTE.*THUNDERX.*\)=y/\1=n/g' config/common_base
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep -E "DPAA|FSLMC|OCTEON|OTX2|THUNDER" config/common_base | grep "=y" || echo "已全部禁用"
已全部禁用
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# make config T=x86_64-native-linuxapp-gcc
Configuration done using x86_64-native-linuxapp-gcc

检查生成的 .config

复制代码
grep -E "DPAA|FSLMC|OCTEON|OTX2|THUNDER" build/.config | grep "=y"

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep -E "DPAA|FSLMC|OCTEON|OTX2|THUNDER" build/.config | grep "=y"
CONFIG_RTE_LIBRTE_COMMON_DPAAX=y
CONFIG_RTE_LIBRTE_DPAA_BUS=y
CONFIG_RTE_LIBRTE_DPAA_MEMPOOL=y
CONFIG_RTE_LIBRTE_DPAA_PMD=y
CONFIG_RTE_LIBRTE_FSLMC_BUS=y
CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL=y
CONFIG_RTE_LIBRTE_DPAA2_PMD=y
CONFIG_RTE_LIBRTE_PMD_DPAA2_SEC=y
CONFIG_RTE_LIBRTE_PMD_DPAA_SEC=y
CONFIG_RTE_LIBRTE_PMD_DPAA_EVENTDEV=y
CONFIG_RTE_LIBRTE_PMD_DPAA2_EVENTDEV=y
CONFIG_RTE_LIBRTE_PMD_DPAA2_CMDIF_RAWDEV=y
CONFIG_RTE_LIBRTE_PMD_DPAA2_QDMA_RAWDEV=y

关键发现make config 生成的 .config 中 ARM 驱动仍然是 =y!说明配置生成过程中有其他来源覆盖了 common_base 的设置。


步骤 7:直接修改 build/.config 并编译

既然 make config 不受 common_base 控制,直接修改生成的 .config

复制代码
sed -i 's/\(CONFIG_RTE_LIBRTE.*DPAA.*\)=y/\1=n/g' build/.config
sed -i 's/\(CONFIG_RTE_LIBRTE.*FSLMC.*\)=y/\1=n/g' build/.config
sed -i 's/\(CONFIG_RTE_LIBRTE.*OCTEONTX.*\)=y/\1=n/g' build/.config
sed -i 's/\(CONFIG_RTE_LIBRTE.*OTX2.*\)=y/\1=n/g' build/.config
sed -i 's/\(CONFIG_RTE_LIBRTE.*THUNDERX.*\)=y/\1=n/g' build/.config

# 验证
grep -E "DPAA|FSLMC|OCTEON|OTX2|THUNDER" build/.config | grep "=y" || echo "配置正确"

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i 's/\(CONFIG_RTE_LIBRTE.*DPAA.*\)=y/\1=n/g' build/.config
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i 's/\(CONFIG_RTE_LIBRTE.*FSLMC.*\)=y/\1=n/g' build/.config
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i 's/\(CONFIG_RTE_LIBRTE.*OCTEONTX.*\)=y/\1=n/g' build/.config
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i 's/\(CONFIG_RTE_LIBRTE.*OTX2.*\)=y/\1=n/g' build/.config
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i 's/\(CONFIG_RTE_LIBRTE.*THUNDERX.*\)=y/\1=n/g' build/.config
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep -E "DPAA|FSLMC|OCTEON|OTX2|THUNDER" build/.config | grep "=y" || echo "配置正确"
配置正确

配置修改成功,开始编译:

复制代码
make -j$(nproc)

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# make -j$(nproc)
...
== Build drivers/mempool/ring
== Build drivers/mempool/octeontx2
== Build drivers/mempool/stack
  CC rte_mempool_bucket.o
  CC rte_mempool_ring.o
  CC otx2_mempool.o
  CC otx2_mempool_irq.o
  CC otx2_mempool_ops.o
  CC otx2_mempool_debug.o
  CC rte_mempool_stack.o
  CC octeontx_fpavf.o
  PMDINFO octeontx_fpavf.o.pmd.c
  CC octeontx_fpavf.o.pmd.o
  LD octeontx_fpavf.o
ld: i386:x86-64 architecture of input file `octeontx_fpavf.o.pmd.o' is incompatible with i386 output
ld: Relocatable linking with relocations from format elf64-x86-64 (octeontx_fpavf.o) to format elf32-i386 (octeontx_fpavf.o.o) is not supported
...
make: *** [all] Error 2

新问题 :现在报错的是 OcteonTX 驱动(另一个 ARM 平台驱动),而且错误信息显示架构不匹配问题依然存在。

这说明有两个问题:

  1. 还有其他 ARM 驱动没有被禁用(OcteonTX)
  2. 架构不匹配的问题根源没有解决

步骤 8:深入分析架构问题

检查架构配置和链接器设置:

复制代码
# 检查架构配置
echo "=== 检查架构设置 ==="
grep -E "^CONFIG_RTE_ARCH" build/.config

# 检查工具链设置
echo "=== 检查工具链设置 ==="
grep -E "TOOLCHAIN|CC=|LD=" build/.config | head -10

# 检查编译器
echo "=== 检查编译器 ==="
which gcc
gcc --version | head -1

# 检查链接器默认架构
echo "=== 检查链接器 ==="
ld -V

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# echo "=== 检查架构设置 ==="
=== 检查架构设置 ===
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep -E "^CONFIG_RTE_ARCH" build/.config
CONFIG_RTE_ARCH="x86_64"
CONFIG_RTE_ARCH_STRICT_ALIGN=n
CONFIG_RTE_ARCH_X86_64=y
CONFIG_RTE_ARCH_X86=y
CONFIG_RTE_ARCH_64=y

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# echo "=== 检查工具链设置 ==="
=== 检查工具链设置 ===
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep -E "TOOLCHAIN|CC=|LD=" build/.config | head -10
# RTE_TOOLCHAIN values are architecture we compile for. directories in mk/toolchain/
CONFIG_RTE_TOOLCHAIN="gcc"
CONFIG_RTE_TOOLCHAIN_GCC=y

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# echo "=== 检查编译器 ==="
=== 检查编译器 ===
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# which gcc
/usr/bin/gcc
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# gcc --version | head -1
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# echo "=== 检查链接器 ==="
=== 检查链接器 ===
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# ld -V
GNU ld (GNU Binutils for Ubuntu) 2.26.1
  Supported emulations:
   elf_i386
   i386linux
   elf_iamcu
   elf32_x86_64
   elf_x86_64
   elf_l1om
   elf_k1om
   i386pep
   i386pe

关键发现

  • 架构配置是正确的:CONFIG_RTE_ARCH="x86_64"
  • 但链接器 ld -V 显示第一个支持的模拟格式是 elf_i386(32 位)!
  • ld 被调用时没有指定输出格式,会使用默认的第一个格式

步骤 9:检查 PMDINFO 链接命令

查看编译脚本中 PMDINFO 相关的链接命令:

复制代码
# 查看 PMDINFO 链接命令
echo "=== PMDINFO 链接命令 ==="
grep -n "PMDINFO" mk/internal/rte.compile-pre.mk

# 查看 LDFLAGS 的实际值
echo "=== 检查编译时 LDFLAGS ==="
grep "LDFLAGS" mk/target/generic/rte.vars.mk | head -10

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# echo "=== PMDINFO 链接命令 ==="
=== PMDINFO 链接命令 ===
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep -n "PMDINFO" mk/internal/rte.compile-pre.mk
62:PMDINFO_GEN = $(RTE_SDK_BIN)/app/dpdk-pmdinfogen $@ $@.pmd.c
63:PMDINFO_CC = $(CC) $(CPPFLAGS) $(CFLAGS) $(EXTRA_CFLAGS) -c -o $@.pmd.o $@.pmd.c
64:PMDINFO_LD = $(CROSS)ld $(LDFLAGS) -r -o $@.o $@.pmd.o $@
65:PMDINFO_TO_O = if grep -q 'RTE_PMD_REGISTER_.*(.*)' $<; then \
66:	echo "$(if $V,$(PMDINFO_GEN),  PMDINFO $@.pmd.c)" && \
67:	$(PMDINFO_GEN) && \
68:	echo "$(if $V,$(PMDINFO_CC),  CC $@.pmd.o)" && \
69:	$(PMDINFO_CC) && \
70:	echo "$(if $V,$(PMDINFO_LD),  LD $@)" && \
71:	$(PMDINFO_LD) && \
77:	$(PMDINFO_TO_O) && \
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# echo "=== 检查编译时 LDFLAGS ==="
=== 检查编译时 LDFLAGS ===
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep "LDFLAGS" mk/target/generic/rte.vars.mk | head -10
15:#   - define TOOLCHAIN_LDFLAGS variable (overridden by cmdline value)
27:#   - define MACHINE_LDFLAGS variable (overridden by cmdline value)
31:#   - can define CPU_LDFLAGS variable (overridden by cmdline value) that
51:#   - define CPU_LDFLAGS variable (overridden by cmdline or previous
63:#   - define EXECENV_LDFLAGS variable (overridden by cmdline)
69:# Don't set CFLAGS/LDFLAGS flags for kernel module, all flags are
81:# merge all LDFLAGS
82:LDFLAGS := $(CPU_LDFLAGS) $(EXECENV_LDFLAGS) $(TOOLCHAIN_LDFLAGS) $(MACHINE_LDFLAGS)
83:LDFLAGS += $(TARGET_LDFLAGS)
91:# LDFLAGS += -L$(RTE_OUTPUT)/lib

关键发现PMDINFO_LD 命令是:

复制代码
$(CROSS)ld $(LDFLAGS) -r -o $@.o $@.pmd.o $@

LDFLAGS 的值来自 CPU_LDFLAGS 等变量的合并。需要检查 CPU_LDFLAGS 的设置。


步骤 10:检查 x86_64 架构的 CPU_LDFLAGS 设置

复制代码
# 查看 x86_64 架构配置文件
cat mk/arch/x86_64/rte.vars.mk

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# cat mk/arch/x86_64/rte.vars.mk
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2010-2014 Intel Corporation

ARCH  ?= x86_64
ARCH_DIR := x86
CROSS ?=

CPU_CFLAGS  ?= -m64
CPU_LDFLAGS ?=
CPU_ASFLAGS ?= -felf64

export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS

RTE_OBJCOPY_TARGET = elf64-x86-64
RTE_OBJCOPY_ARCH = i386:x86-64

export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH

找到根因了!

CPU_LDFLAGS ?= 后面是空的!这意味着:

  • ld 被调用时,没有 -m elf_x86_64 参数
  • 链接器使用默认的第一个模拟格式 elf_i386(32 位)
  • 导致输出格式是 32 位,但输入是 64 位,产生架构不匹配错误

步骤 11:修复 CPU_LDFLAGS

修改 mk/arch/x86_64/rte.vars.mk,添加链接器格式参数:

复制代码
# 备份原文件
cp mk/arch/x86_64/rte.vars.mk mk/arch/x86_64/rte.vars.mk.bak

# 修改 CPU_LDFLAGS,添加 -m elf_x86_64
sed -i 's/CPU_LDFLAGS ?=/CPU_LDFLAGS ?= -m elf_x86_64/' mk/arch/x86_64/rte.vars.mk

# 验证修改
echo "=== 验证修改 ==="
grep "CPU_LDFLAGS" mk/arch/x86_64/rte.vars.mk

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# cp mk/arch/x86_64/rte.vars.mk mk/arch/x86_64/rte.vars.mk.bak
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i 's/CPU_LDFLAGS ?=/CPU_LDFLAGS ?= -m elf_x86_64/' mk/arch/x86_64/rte.vars.mk
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# echo "=== 验证修改 ==="
=== 验证修改 ===
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep "CPU_LDFLAGS" mk/arch/x86_64/rte.vars.mk
#   - define CPU_LDFLAGS variable (overridden by cmdline or previous
CPU_LDFLAGS ?= -m elf_x86_64
export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS

修改成功!现在清理并重新编译:

复制代码
# 清理之前失败的编译产物
make clean 2>/dev/null

# 重新编译
make -j$(nproc)

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# make clean 2>/dev/null
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# make -j$(nproc)
...
  CC test_ipsec.o
  MKRES test_resource_c.res.o
  LD test
/usr/bin/ld: cannot find -lnuma
collect2: error: ld returned 1 exit status
...
make: *** [all] Error 2

新进展 :架构不匹配问题解决了!现在是缺少 libnuma 库。


步骤 12:安装 numa 库

复制代码
# 检查 numa 库是否安装
dpkg -l | grep numa

# 检查 numa 库位置
ldconfig -p | grep numa

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# dpkg -l | grep numa
ii  libnuma-dev:i386                          2.0.11-1ubuntu1.1                               i386         Development files for libnuma
ii  libnuma1:i386                             2.0.11-1ubuntu1.1                               i386         Libraries for controlling NUMA policy

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# ldconfig -p | grep numa
	libnuma.so.1 (libc6) => /usr/lib/i386-linux-gnu/libnuma.so.1
	libnuma.so (libc6) => /usr/lib/i386-linux-gnu/libnuma.so

问题 :系统安装的是 32 位 (i386) 的 numa 库,但需要 64 位 (amd64) 的版本。

尝试安装 64 位版本:

复制代码
# 安装 64 位 numa 开发库
sudo apt-get install -y libnuma-dev:amd64 libnuma1:amd64

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sudo apt-get install -y libnuma-dev:amd64 libnuma1:amd64
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package libnuma-dev:amd64

问题:系统没有启用 amd64 架构支持。


步骤 13:启用 amd64 架构并安装 64 位 numa 库

复制代码
# 启用 amd64 架构
sudo dpkg --add-architecture amd64
sudo apt-get update

# 安装 64 位 numa 库
sudo apt-get install -y libnuma-dev:amd64 libnuma1:amd64

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sudo dpkg --add-architecture amd64
root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sudo apt-get update
Hit:1 http://us.archive.ubuntu.com/ubuntu xenial InRelease                                                   
Hit:2 http://us.archive.ubuntu.com/ubuntu xenial-updates InRelease      
Hit:3 http://us.archive.ubuntu.com/ubuntu xenial-backports InRelease    
Hit:4 http://security.ubuntu.com/ubuntu xenial-security InRelease       
Get:5 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 Packages [1,201 kB]
...
Fetched 13.0 MB in 11s (1,096 kB/s)                                                                          
Reading package lists... Done

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sudo apt-get install -y libnuma-dev:amd64 libnuma1:amd64
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  gcc-6-base:amd64 libc6:amd64 libc6-dev:amd64 libgcc1:amd64 linux-libc-dev:amd64
...
Setting up libnuma-dev:amd64 (2.0.11-1ubuntu1.1) ...
Setting up libgcc1:amd64 (1:6.0.1-0ubuntu1) ...
Processing triggers for libc-bin (2.23-0ubuntu11.3) ...

验证 64 位库安装成功:

复制代码
ldconfig -p | grep numa

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# ldconfig -p | grep numa
	libnuma.so.1 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libnuma.so.1
	libnuma.so.1 (libc6) => /usr/lib/i386-linux-gnu/libnuma.so.1
	libnuma.so (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libnuma.so
	libnuma.so (libc6) => /usr/lib/i386-linux-gnu/libnuma.so

现在有 64 位和 32 位两个版本了。


步骤 14:继续编译

复制代码
make -j$(nproc)

root@ubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# make -j$(nproc)
...
  CC comp_perf_options_parse.o
  CC comp_perf_test_common.o
  CC comp_perf_test_verify.o
  CC comp_perf_test_benchmark.o
  CC main.o
  CC main.o
  CC evt_main.o
  CC evt_options.o
  CC cperf_ops.o
  CC cperf_options_parsing.o
  CC cperf_test_vectors.o
  CC evt_test.o
  CC cperf_test_throughput.o
  LD dpdk-test-compress-perf
  INSTALL-APP test
  INSTALL-MAP test.map
  ...
  LD dpdk-test-crypto-perf
  LD dpdk-test-eventdev
  INSTALL-APP dpdk-test-crypto-perf
  INSTALL-MAP dpdk-test-crypto-perf.map
  INSTALL-APP dpdk-test-eventdev
  INSTALL-MAP dpdk-test-eventdev.map
Build complete [x86_64-native-linuxapp-gcc]

编译成功!


三、问题总结

3.1 问题根因

本次编译失败涉及 三个层面的问题

层面 问题 影响
配置层 DPDK 19.08 默认配置包含 ARM 平台驱动 尝试编译不兼容的 ARM 驱动
工具链层 CPU_LDFLAGS 为空,链接器使用 32 位默认格式 架构不匹配错误
依赖层 系统仅安装 32 位 numa 库 链接时找不到 64 位库

3.2 解决方案总结

复制代码
# ===== 完整修复脚本 =====

cd /home/jerry/Desktop/dpdk-stable-19.08.2

# 1. 清理所有编译产物
rm -rf build x86_64-native-linuxapp-gcc

# 2. 禁用 ARM 平台驱动
sed -i 's/\(CONFIG_RTE_LIBRTE.*DPAA.*\)=y/\1=n/g' config/common_base
sed -i 's/\(CONFIG_RTE_LIBRTE.*FSLMC.*\)=y/\1=n/g' config/common_base
sed -i 's/\(CONFIG_RTE_LIBRTE.*OCTEONTX.*\)=y/\1=n/g' config/common_base
sed -i 's/\(CONFIG_RTE_LIBRTE.*OTX2.*\)=y/\1=n/g' config/common_base
sed -i 's/\(CONFIG_RTE_LIBRTE.*THUNDERX.*\)=y/\1=n/g' config/common_base

# 3. 修复链接器架构问题
sed -i 's/CPU_LDFLAGS ?=$/CPU_LDFLAGS ?= -m elf_x86_64/' mk/arch/x86_64/rte.vars.mk

# 4. 安装 64 位 numa 库
sudo dpkg --add-architecture amd64
sudo apt-get update
sudo apt-get install -y libnuma-dev:amd64 libnuma1:amd64

# 5. 生成配置
make config T=x86_64-native-linuxapp-gcc

# 6. 确保 build/.config 中 ARM 驱动被禁用
sed -i 's/\(CONFIG_RTE_LIBRTE.*DPAA.*\)=y/\1=n/g' build/.config
sed -i 's/\(CONFIG_RTE_LIBRTE.*FSLMC.*\)=y/\1=n/g' build/.config
sed -i 's/\(CONFIG_RTE_LIBRTE.*OCTEONTX.*\)=y/\1=n/g' build/.config
sed -i 's/\(CONFIG_RTE_LIBRTE.*OTX2.*\)=y/\1=n/g' build/.config

# 7. 编译
make -j$(nproc)

3.3 为什么博客里没有这个问题?

可能的原因:

  1. DPDK 版本差异:不同版本的默认配置不同
  2. 系统环境差异:其他环境可能已安装正确依赖
  3. 编译方式差异:可能使用了 meson 而非 make
  4. Ubuntu 版本差异:不同 Ubuntu 版本的 binutils 默认行为可能不同

四、技术细节说明

4.1 PMDINFO 链接过程

DPDK 编译过程中的 PMDINFO 步骤:

复制代码
源文件 (.c)
    ↓ gcc -c
目标文件
    ↓ dpdk-pmdinfogen
PMD 信息文件
    ↓ gcc -c
PMD 目标文件
    ↓ ld -r        ← 这里需要指定正确的输出格式
合并目标文件

ld -r 是可重定位链接,不会生成最终可执行文件,只是合并多个目标文件。

4.2 链接器模拟格式

ld -V 输出的模拟格式列表:

格式 说明
elf_i386 32 位 x86 ELF
elf_x86_64 64 位 x86_64 ELF
elf32_x86_64 32 位兼容的 x86_64

当不指定 -m 参数时,链接器使用列表中的第一个格式(elf_i386),这就是问题所在。

4.3 DPDK 配置优先级

复制代码
config/defconfig_x86_64-native-linuxapp-gcc  (最高优先级)
    +
config/common_base
    +
config/common_linux
    ↓
build/.config  (实际使用的配置)

某些配置在 defconfig 文件中被显式设置,会覆盖 common_base 的值。

相关推荐
微学AI2 小时前
复杂查询中 JOIN 条件下推失败导致的性能瓶颈-金仓数据库
数据库
cyforkk2 小时前
数据库里的隐形守卫:通俗易懂理解 RLS(行级安全)
数据库·安全
zxrhhm2 小时前
Oracle一般而言standby redo日志文件组数要比primary数据库的online redo日志文件组数至少多一个,为什么?
数据库·oracle
小鸡脚来咯2 小时前
SQL常用函数
数据库·sql
道长没有道观2 小时前
mysql database learn
数据库·mysql
code_li2 小时前
多数据高性能 同步 到本地数据库表里
数据库
一个天蝎座 白勺 程序猿2 小时前
Apache IoTDB(18):IoTDB时序数据库的数据同步之Pipe机制与插件同步指南
数据库·apache·时序数据库·iotdb
颜颜yan_2 小时前
告别“子查询性能陷阱“:金仓数据库智能下推优化器的设计与实测
数据库·oracle
sdanss2 小时前
MySQL——表的约束
数据库·mysql