一、问题背景1.1 环境信息项目版本操作系统Ubuntu 16.04 LTSDPDK 版本19.08.2GCC 版本5.4.0Binutils2.26.1目标架构x86_641.2 原始错误信息使用./usertools/dpdk-setup.sh编译 DPDK 时遇到以下错误CC portal/dpaa2_hw_dpio.o PMDINFO portal/dpaa2_hw_dpio.o.pmd.c CC portal/dpaa2_hw_dpio.o.pmd.o LD portal/dpaa2_hw_dpio.o ld: i386:x86-64 architecture of input file portal/dpaa2_hw_dpio.o.pmd.o is incompatible with i386 output ld: Relocatable linking with relocations from format elf64-x86-64 (portal/dpaa2_hw_dpio.o) to format elf32-i386 (portal/dpaa2_hw_dpio.o.o) is not supported /home/jerry/Desktop/dpdk-stable-19.08.2/mk/internal/rte.compile-pre.mk:114: recipe for target portal/dpaa2_hw_dpio.o failed make[6]: *** [portal/dpaa2_hw_dpio.o] Error 1 ... RTE_TARGET exported as x86_64-native-linuxapp-gcc错误关键信息输入文件架构elf64-x86-6464 位输出格式elf32-i38632 位问题文件portal/dpaa2_hw_dpio.o属于 FSLMC/DPAA2 驱动目标架构设置x86_64-native-linuxapp-gcc看起来是正确的二、调试过程步骤 1尝试修改配置文件禁用 ARM 驱动首先尝试修改config/common_linuxapprootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2/usertools# sed -i s/CONFIG_RTE_LIBRTE_FSLMC_BUSy/CONFIG_RTE_LIBRTE_FSLMC_BUSn/ config/common_linuxapp sed: cant read config/common_linuxapp: No such file or directory rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2/usertools# sed -i s/CONFIG_RTE_LIBRTE_DPAA2_PMDy/CONFIG_RTE_LIBRTE_DPAA2_PMDn/ config/common_linuxapp sed: cant read config/common_linuxapp: No such file or directory问题配置文件路径不对当前在usertools目录下且 DPDK 19.08 使用的是config/common_base。步骤 2回到正确目录修改正确的配置文件cd /home/jerry/Desktop/dpdk-stable-19.08.2 ls config/rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# cd /home/jerry/Desktop/dpdk-stable-19.08.2 rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# ls config/ arm64-armada-linuxapp-gcc defconfig_x86_64-native-linuxapp-clang arm64-armada-linux-gcc defconfig_x86_64-native-linuxapp-gcc arm64-armv8a-linuxapp-clang defconfig_x86_64-native-linuxapp-icc arm64-armv8a-linuxapp-gcc defconfig_x86_64-native-linux-clang arm64-armv8a-linux-clang defconfig_x86_64-native-linux-gcc arm64-armv8a-linux-gcc defconfig_x86_64-native-linux-icc arm64-bluefield-linuxapp-gcc defconfig_x86_x32-native-linuxapp-gcc arm64-bluefield-linux-gcc defconfig_x86_x32-native-linux-gcc arm64-dpaa2-linuxapp-gcc meson.build arm64-dpaa2-linux-gcc ppc_64-power8-linuxapp-gcc arm64-dpaa-linuxapp-gcc ppc_64-power8-linux-gcc arm64-dpaa-linux-gcc rte_config.h arm64-octeontx2-linuxapp-gcc rte_compatibility_defines.h arm64-octeontx2-linux-gcc rte_config.h.in arm64-stingray-linuxapp-gcc x86_64-native-bsdapp-clang arm64-stingray-linux-gcc x86_64-native-bsdapp-gcc arm64-thunderx2-linuxapp-gcc x86_64-native-freebsd-clang arm64-thunderx2-linux-gcc x86_64-native-freebsd-gcc arm64-thunderx-linuxapp-gcc x86_64-native-linuxapp-clang arm64-thunderx-linux-gcc x86_64-native-linuxapp-gcc arm64-xgene1-linuxapp-gcc x86_64-native-linuxapp-icc arm64-xgene1-linux-gcc x86_64-native-linux-clang arm-armv7a-linuxapp-gcc x86_64-native-linux-gcc arm-armv7a-linux-gcc x86_64-native-linux-icc common_base x86_x32-native-linuxapp-gcc common_linux x86_x32-native-linux-gcc defconfig_i686-native-linuxapp-gcc defconfig_i686-native-linuxapp-icc defconfig_i686-native-linux-gcc defconfig_i686-native-linux-icc找到common_base文件查看 DPAA/FSLMC 配置状态grep -E FSLMC|DPAA2 config/common_base | head -20rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep -E FSLMC|DPAA2 config/common_base | head -20 # Compile NXP DPAA2 FSL-MC Bus CONFIG_RTE_LIBRTE_FSLMC_BUSn # Compile Support Libraries for NXP DPAA2 CONFIG_RTE_LIBRTE_DPAA2_MEMPOOLn CONFIG_RTE_LIBRTE_DPAA2_USE_PHYS_IOVAy # Compile burst-oriented NXP DPAA2 PMD driver CONFIG_RTE_LIBRTE_DPAA2_PMDn CONFIG_RTE_LIBRTE_DPAA2_DEBUG_DRIVERn # Compile NXP DPAA2 crypto sec driver for CAAM HW CONFIG_RTE_LIBRTE_PMD_DPAA2_SECn # Compile PMD for NXP DPAA2 event device CONFIG_RTE_LIBRTE_PMD_DPAA2_EVENTDEVn # Compile PMD for NXP DPAA2 CMDIF raw device CONFIG_RTE_LIBRTE_PMD_DPAA2_CMDIF_RAWDEVn # Compile PMD for NXP DPAA2 QDMA raw device CONFIG_RTE_LIBRTE_PMD_DPAA2_QDMA_RAWDEVn配置文件中已经是n说明问题不在这里。步骤 3清理并重新编译# 清理 rm -rf build rm -f .config # 重新配置并编译 make config Tx86_64-native-linuxapp-gcc make -j$(nproc)rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# rm -rf build rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# rm -f .config rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# make config Tx86_64-native-linuxapp-gcc Configuration done using x86_64-native-linuxapp-gcc rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# make -j$(nproc) ... CC portal/dpaa2_hw_dpio.o PMDINFO portal/dpaa2_hw_dpio.o.pmd.c CC portal/dpaa2_hw_dpio.o.pmd.o LD portal/dpaa2_hw_dpio.o ld: i386:x86-64 architecture of input file portal/dpaa2_hw_dpio.o.pmd.o is incompatible with i386 output ld: Relocatable linking with relocations from format elf64-x86-64 (portal/dpaa2_hw_dpio.o) to format elf32-i386 (portal/dpaa2_hw_dpio.o.o) is not supported /home/jerry/Desktop/dpdk-stable-19.08.2/mk/internal/rte.compile-pre.mk:114: recipe for target portal/dpaa2_hw_dpio.o failed make[4]: *** [portal/dpaa2_hw_dpio.o] Error 1 ... make: *** [all] Error 2问题依旧即使config/common_base中已经设置为n编译时仍然在尝试编译 FSLMC 驱动。步骤 4检查 build/.config 实际配置关键发现make config生成的build/.config才是实际使用的配置文件。grep -E FSLMC|DPAA build/.config | grep -v ^#rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep -E FSLMC|DPAA build/.config | grep -v ^# CONFIG_RTE_LIBRTE_COMMON_DPAAXy CONFIG_RTE_LIBRTE_DPAA_BUSy CONFIG_RTE_LIBRTE_DPAA_MEMPOOLy CONFIG_RTE_LIBRTE_DPAA_PMDy CONFIG_RTE_LIBRTE_DPAA_HWDEBUGn CONFIG_RTE_LIBRTE_FSLMC_BUSy CONFIG_RTE_LIBRTE_DPAA2_MEMPOOLy CONFIG_RTE_LIBRTE_DPAA2_USE_PHYS_IOVAy CONFIG_RTE_LIBRTE_DPAA2_PMDy CONFIG_RTE_LIBRTE_DPAA2_DEBUG_DRIVERn CONFIG_RTE_LIBRTE_PMD_DPAA2_SECy CONFIG_RTE_LIBRTE_PMD_DPAA_SECy CONFIG_RTE_LIBRTE_DPAA_MAX_CRYPTODEV4 CONFIG_RTE_LIBRTE_PMD_DPAA_EVENTDEVy CONFIG_RTE_LIBRTE_PMD_DPAA2_EVENTDEVy CONFIG_RTE_LIBRTE_PMD_DPAA2_CMDIF_RAWDEVy CONFIG_RTE_LIBRTE_PMD_DPAA2_QDMA_RAWDEVy关键发现build/.config中大量的 ARM 驱动配置仍然是yconfig/common_base的修改没有生效。步骤 5直接修改 build/.configsed -i s/\(CONFIG_RTE_LIBRTE.*DPAA.*\)y/\1n/g build/.config sed -i s/\(CONFIG_RTE_LIBRTE.*FSLMC.*\)y/\1n/g build/.config # 验证 grep -E FSLMC|DPAA build/.config | grep yrootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i s/\(CONFIG_RTE_LIBRTE.*DPAA.*\)y/\1n/g build/.config rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i s/\(CONFIG_RTE_LIBRTE.*FSLMC.*\)y/\1n/g build/.config rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep -E FSLMC|DPAA build/.config | grep y CONFIG_RTE_LIBRTE_DPAA_MEMPOOLy CONFIG_RTE_LIBRTE_DPAA_PMDy CONFIG_RTE_LIBRTE_DPAA2_MEMPOOLy CONFIG_RTE_LIBRTE_DPAA2_USE_PHYS_IOVAy CONFIG_RTE_LIBRTE_DPAA2_PMDy CONFIG_RTE_LIBRTE_PMD_DPAA2_SECy CONFIG_RTE_LIBRTE_PMD_DPAA_SECy CONFIG_RTE_LIBRTE_PMD_DPAA_EVENTDEVy CONFIG_RTE_LIBRTE_PMD_DPAA2_EVENTDEVy CONFIG_RTE_LIBRTE_PMD_DPAA2_CMDIF_RAWDEVy CONFIG_RTE_LIBRTE_PMD_DPAA2_QDMA_RAWDEVy问题sed 正则没有完全匹配所有配置项有些配置项名称不包含LIBRTE。步骤 6彻底禁用所有 ARM 相关配置需要更彻底地修改# 彻底删除 build 目录重新开始 rm -rf build # 先修改 config/common_base sed -i s/\(CONFIG_RTE_LIBRTE.*DPAA.*\)y/\1n/g config/common_base sed -i s/\(CONFIG_RTE_LIBRTE.*FSLMC.*\)y/\1n/g config/common_base sed -i s/\(CONFIG_RTE_LIBRTE.*OCTEONTX.*\)y/\1n/g config/common_base sed -i s/\(CONFIG_RTE_LIBRTE.*OTX2.*\)y/\1n/g config/common_base sed -i s/\(CONFIG_RTE_LIBRTE.*THUNDERX.*\)y/\1n/g config/common_base # 验证 grep -E DPAA|FSLMC|OCTEON|OTX2|THUNDER config/common_base | grep y || echo 已全部禁用 # 重新配置 make config Tx86_64-native-linuxapp-gccrootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# rm -rf build rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i s/\(CONFIG_RTE_LIBRTE.*DPAA.*\)y/\1n/g config/common_base rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i s/\(CONFIG_RTE_LIBRTE.*FSLMC.*\)y/\1n/g config/common_base rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i s/\(CONFIG_RTE_LIBRTE.*OCTEONTX.*\)y/\1n/g config/common_base rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i s/\(CONFIG_RTE_LIBRTE.*OTX2.*\)y/\1n/g config/common_base rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i s/\(CONFIG_RTE_LIBRTE.*THUNDERX.*\)y/\1n/g config/common_base rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep -E DPAA|FSLMC|OCTEON|OTX2|THUNDER config/common_base | grep y || echo 已全部禁用 已全部禁用 rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# make config Tx86_64-native-linuxapp-gcc Configuration done using x86_64-native-linuxapp-gcc检查生成的.configgrep -E DPAA|FSLMC|OCTEON|OTX2|THUNDER build/.config | grep yrootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep -E DPAA|FSLMC|OCTEON|OTX2|THUNDER build/.config | grep y CONFIG_RTE_LIBRTE_COMMON_DPAAXy CONFIG_RTE_LIBRTE_DPAA_BUSy CONFIG_RTE_LIBRTE_DPAA_MEMPOOLy CONFIG_RTE_LIBRTE_DPAA_PMDy CONFIG_RTE_LIBRTE_FSLMC_BUSy CONFIG_RTE_LIBRTE_DPAA2_MEMPOOLy CONFIG_RTE_LIBRTE_DPAA2_PMDy CONFIG_RTE_LIBRTE_PMD_DPAA2_SECy CONFIG_RTE_LIBRTE_PMD_DPAA_SECy CONFIG_RTE_LIBRTE_PMD_DPAA_EVENTDEVy CONFIG_RTE_LIBRTE_PMD_DPAA2_EVENTDEVy CONFIG_RTE_LIBRTE_PMD_DPAA2_CMDIF_RAWDEVy CONFIG_RTE_LIBRTE_PMD_DPAA2_QDMA_RAWDEVy关键发现make config生成的.config中 ARM 驱动仍然是y说明配置生成过程中有其他来源覆盖了common_base的设置。步骤 7直接修改 build/.config 并编译既然make config不受common_base控制直接修改生成的.configsed -i s/\(CONFIG_RTE_LIBRTE.*DPAA.*\)y/\1n/g build/.config sed -i s/\(CONFIG_RTE_LIBRTE.*FSLMC.*\)y/\1n/g build/.config sed -i s/\(CONFIG_RTE_LIBRTE.*OCTEONTX.*\)y/\1n/g build/.config sed -i s/\(CONFIG_RTE_LIBRTE.*OTX2.*\)y/\1n/g build/.config sed -i s/\(CONFIG_RTE_LIBRTE.*THUNDERX.*\)y/\1n/g build/.config # 验证 grep -E DPAA|FSLMC|OCTEON|OTX2|THUNDER build/.config | grep y || echo 配置正确rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i s/\(CONFIG_RTE_LIBRTE.*DPAA.*\)y/\1n/g build/.config rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i s/\(CONFIG_RTE_LIBRTE.*FSLMC.*\)y/\1n/g build/.config rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i s/\(CONFIG_RTE_LIBRTE.*OCTEONTX.*\)y/\1n/g build/.config rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i s/\(CONFIG_RTE_LIBRTE.*OTX2.*\)y/\1n/g build/.config rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i s/\(CONFIG_RTE_LIBRTE.*THUNDERX.*\)y/\1n/g build/.config rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep -E DPAA|FSLMC|OCTEON|OTX2|THUNDER build/.config | grep y || echo 配置正确 配置正确配置修改成功开始编译make -j$(nproc)rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# make -j$(nproc) ... Build drivers/mempool/ring Build drivers/mempool/octeontx2 Build drivers/mempool/stack CC rte_mempool_bucket.o CC rte_mempool_ring.o CC otx2_mempool.o CC otx2_mempool_irq.o CC otx2_mempool_ops.o CC otx2_mempool_debug.o CC rte_mempool_stack.o CC octeontx_fpavf.o PMDINFO octeontx_fpavf.o.pmd.c CC octeontx_fpavf.o.pmd.o LD octeontx_fpavf.o ld: i386:x86-64 architecture of input file octeontx_fpavf.o.pmd.o is incompatible with i386 output ld: Relocatable linking with relocations from format elf64-x86-64 (octeontx_fpavf.o) to format elf32-i386 (octeontx_fpavf.o.o) is not supported ... make: *** [all] Error 2新问题现在报错的是OcteonTX驱动另一个 ARM 平台驱动而且错误信息显示架构不匹配问题依然存在。这说明有两个问题还有其他 ARM 驱动没有被禁用OcteonTX架构不匹配的问题根源没有解决步骤 8深入分析架构问题检查架构配置和链接器设置# 检查架构配置 echo 检查架构设置 grep -E ^CONFIG_RTE_ARCH build/.config # 检查工具链设置 echo 检查工具链设置 grep -E TOOLCHAIN|CC|LD build/.config | head -10 # 检查编译器 echo 检查编译器 which gcc gcc --version | head -1 # 检查链接器默认架构 echo 检查链接器 ld -Vrootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# echo 检查架构设置 检查架构设置 rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep -E ^CONFIG_RTE_ARCH build/.config CONFIG_RTE_ARCHx86_64 CONFIG_RTE_ARCH_STRICT_ALIGNn CONFIG_RTE_ARCH_X86_64y CONFIG_RTE_ARCH_X86y CONFIG_RTE_ARCH_64y rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# echo 检查工具链设置 检查工具链设置 rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep -E TOOLCHAIN|CC|LD build/.config | head -10 # RTE_TOOLCHAIN values are architecture we compile for. directories in mk/toolchain/ CONFIG_RTE_TOOLCHAINgcc CONFIG_RTE_TOOLCHAIN_GCCy rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# echo 检查编译器 检查编译器 rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# which gcc /usr/bin/gcc rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# gcc --version | head -1 gcc (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609 rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# echo 检查链接器 检查链接器 rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# ld -V GNU ld (GNU Binutils for Ubuntu) 2.26.1 Supported emulations: elf_i386 i386linux elf_iamcu elf32_x86_64 elf_x86_64 elf_l1om elf_k1om i386pep i386pe关键发现架构配置是正确的CONFIG_RTE_ARCHx86_64但链接器ld -V显示第一个支持的模拟格式是elf_i38632 位当ld被调用时没有指定输出格式会使用默认的第一个格式步骤 9检查 PMDINFO 链接命令查看编译脚本中 PMDINFO 相关的链接命令# 查看 PMDINFO 链接命令 echo PMDINFO 链接命令 grep -n PMDINFO mk/internal/rte.compile-pre.mk # 查看 LDFLAGS 的实际值 echo 检查编译时 LDFLAGS grep LDFLAGS mk/target/generic/rte.vars.mk | head -10rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# echo PMDINFO 链接命令 PMDINFO 链接命令 rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep -n PMDINFO mk/internal/rte.compile-pre.mk 62:PMDINFO_GEN $(RTE_SDK_BIN)/app/dpdk-pmdinfogen $ $.pmd.c 63:PMDINFO_CC $(CC) $(CPPFLAGS) $(CFLAGS) $(EXTRA_CFLAGS) -c -o $.pmd.o $.pmd.c 64:PMDINFO_LD $(CROSS)ld $(LDFLAGS) -r -o $.o $.pmd.o $ 65:PMDINFO_TO_O if grep -q RTE_PMD_REGISTER_.*(.*) $; then \ 66: echo $(if $V,$(PMDINFO_GEN), PMDINFO $.pmd.c) \ 67: $(PMDINFO_GEN) \ 68: echo $(if $V,$(PMDINFO_CC), CC $.pmd.o) \ 69: $(PMDINFO_CC) \ 70: echo $(if $V,$(PMDINFO_LD), LD $) \ 71: $(PMDINFO_LD) \ 77: $(PMDINFO_TO_O) \ rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# echo 检查编译时 LDFLAGS 检查编译时 LDFLAGS rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep LDFLAGS mk/target/generic/rte.vars.mk | head -10 15:# - define TOOLCHAIN_LDFLAGS variable (overridden by cmdline value) 27:# - define MACHINE_LDFLAGS variable (overridden by cmdline value) 31:# - can define CPU_LDFLAGS variable (overridden by cmdline value) that 51:# - define CPU_LDFLAGS variable (overridden by cmdline or previous 63:# - define EXECENV_LDFLAGS variable (overridden by cmdline) 69:# Dont set CFLAGS/LDFLAGS flags for kernel module, all flags are 81:# merge all LDFLAGS 82:LDFLAGS : $(CPU_LDFLAGS) $(EXECENV_LDFLAGS) $(TOOLCHAIN_LDFLAGS) $(MACHINE_LDFLAGS) 83:LDFLAGS $(TARGET_LDFLAGS) 91:# LDFLAGS -L$(RTE_OUTPUT)/lib关键发现PMDINFO_LD命令是$(CROSS)ld $(LDFLAGS) -r -o $.o $.pmd.o $LDFLAGS的值来自CPU_LDFLAGS等变量的合并。需要检查CPU_LDFLAGS的设置。步骤 10检查 x86_64 架构的 CPU_LDFLAGS 设置# 查看 x86_64 架构配置文件 cat mk/arch/x86_64/rte.vars.mkrootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# cat mk/arch/x86_64/rte.vars.mk # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2010-2014 Intel Corporation ARCH ? x86_64 ARCH_DIR : x86 CROSS ? CPU_CFLAGS ? -m64 CPU_LDFLAGS ? CPU_ASFLAGS ? -felf64 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS RTE_OBJCOPY_TARGET elf64-x86-64 RTE_OBJCOPY_ARCH i386:x86-64 export RTE_OBJCOPY_TARGET RTE_OBJCOPY_ARCH找到根因了CPU_LDFLAGS ?后面是空的这意味着当ld被调用时没有-m elf_x86_64参数链接器使用默认的第一个模拟格式elf_i38632 位导致输出格式是 32 位但输入是 64 位产生架构不匹配错误步骤 11修复 CPU_LDFLAGS修改mk/arch/x86_64/rte.vars.mk添加链接器格式参数# 备份原文件 cp mk/arch/x86_64/rte.vars.mk mk/arch/x86_64/rte.vars.mk.bak # 修改 CPU_LDFLAGS添加 -m elf_x86_64 sed -i s/CPU_LDFLAGS ?/CPU_LDFLAGS ? -m elf_x86_64/ mk/arch/x86_64/rte.vars.mk # 验证修改 echo 验证修改 grep CPU_LDFLAGS mk/arch/x86_64/rte.vars.mkrootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# cp mk/arch/x86_64/rte.vars.mk mk/arch/x86_64/rte.vars.mk.bak rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sed -i s/CPU_LDFLAGS ?/CPU_LDFLAGS ? -m elf_x86_64/ mk/arch/x86_64/rte.vars.mk rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# echo 验证修改 验证修改 rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# grep CPU_LDFLAGS mk/arch/x86_64/rte.vars.mk # - define CPU_LDFLAGS variable (overridden by cmdline or previous CPU_LDFLAGS ? -m elf_x86_64 export ARCH CROSS CPU_CFLAGS CPU_LDFLAGS CPU_ASFLAGS修改成功现在清理并重新编译# 清理之前失败的编译产物 make clean 2/dev/null # 重新编译 make -j$(nproc)rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# make clean 2/dev/null rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# make -j$(nproc) ... CC test_ipsec.o MKRES test_resource_c.res.o LD test /usr/bin/ld: cannot find -lnuma collect2: error: ld returned 1 exit status ... make: *** [all] Error 2新进展架构不匹配问题解决了现在是缺少libnuma库。步骤 12安装 numa 库# 检查 numa 库是否安装 dpkg -l | grep numa # 检查 numa 库位置 ldconfig -p | grep numarootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# dpkg -l | grep numa ii libnuma-dev:i386 2.0.11-1ubuntu1.1 i386 Development files for libnuma ii libnuma1:i386 2.0.11-1ubuntu1.1 i386 Libraries for controlling NUMA policy rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# ldconfig -p | grep numa libnuma.so.1 (libc6) /usr/lib/i386-linux-gnu/libnuma.so.1 libnuma.so (libc6) /usr/lib/i386-linux-gnu/libnuma.so问题系统安装的是32 位 (i386)的 numa 库但需要64 位 (amd64)的版本。尝试安装 64 位版本# 安装 64 位 numa 开发库 sudo apt-get install -y libnuma-dev:amd64 libnuma1:amd64rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sudo apt-get install -y libnuma-dev:amd64 libnuma1:amd64 Reading package lists... Done Building dependency tree Reading state information... Done E: Unable to locate package libnuma-dev:amd64问题系统没有启用 amd64 架构支持。步骤 13启用 amd64 架构并安装 64 位 numa 库# 启用 amd64 架构 sudo dpkg --add-architecture amd64 sudo apt-get update # 安装 64 位 numa 库 sudo apt-get install -y libnuma-dev:amd64 libnuma1:amd64rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sudo dpkg --add-architecture amd64 rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sudo apt-get update Hit:1 http://us.archive.ubuntu.com/ubuntu xenial InRelease Hit:2 http://us.archive.ubuntu.com/ubuntu xenial-updates InRelease Hit:3 http://us.archive.ubuntu.com/ubuntu xenial-backports InRelease Hit:4 http://security.ubuntu.com/ubuntu xenial-security InRelease Get:5 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 Packages [1,201 kB] ... Fetched 13.0 MB in 11s (1,096 kB/s) Reading package lists... Done rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# sudo apt-get install -y libnuma-dev:amd64 libnuma1:amd64 Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: gcc-6-base:amd64 libc6:amd64 libc6-dev:amd64 libgcc1:amd64 linux-libc-dev:amd64 ... Setting up libnuma-dev:amd64 (2.0.11-1ubuntu1.1) ... Setting up libgcc1:amd64 (1:6.0.1-0ubuntu1) ... Processing triggers for libc-bin (2.23-0ubuntu11.3) ...验证 64 位库安装成功ldconfig -p | grep numarootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# ldconfig -p | grep numa libnuma.so.1 (libc6,x86-64) /usr/lib/x86_64-linux-gnu/libnuma.so.1 libnuma.so.1 (libc6) /usr/lib/i386-linux-gnu/libnuma.so.1 libnuma.so (libc6,x86-64) /usr/lib/x86_64-linux-gnu/libnuma.so libnuma.so (libc6) /usr/lib/i386-linux-gnu/libnuma.so现在有 64 位和 32 位两个版本了。步骤 14继续编译make -j$(nproc)rootubuntu:/home/jerry/Desktop/dpdk-stable-19.08.2# make -j$(nproc) ... CC comp_perf_options_parse.o CC comp_perf_test_common.o CC comp_perf_test_verify.o CC comp_perf_test_benchmark.o CC main.o CC main.o CC evt_main.o CC evt_options.o CC cperf_ops.o CC cperf_options_parsing.o CC cperf_test_vectors.o CC evt_test.o CC cperf_test_throughput.o LD dpdk-test-compress-perf INSTALL-APP test INSTALL-MAP test.map ... LD dpdk-test-crypto-perf LD dpdk-test-eventdev INSTALL-APP dpdk-test-crypto-perf INSTALL-MAP dpdk-test-crypto-perf.map INSTALL-APP dpdk-test-eventdev INSTALL-MAP dpdk-test-eventdev.map Build complete [x86_64-native-linuxapp-gcc]编译成功三、问题总结3.1 问题根因本次编译失败涉及三个层面的问题层面问题影响配置层DPDK 19.08 默认配置包含 ARM 平台驱动尝试编译不兼容的 ARM 驱动工具链层CPU_LDFLAGS为空链接器使用 32 位默认格式架构不匹配错误依赖层系统仅安装 32 位 numa 库链接时找不到 64 位库3.2 解决方案总结# 完整修复脚本 cd /home/jerry/Desktop/dpdk-stable-19.08.2 # 1. 清理所有编译产物 rm -rf build x86_64-native-linuxapp-gcc # 2. 禁用 ARM 平台驱动 sed -i s/\(CONFIG_RTE_LIBRTE.*DPAA.*\)y/\1n/g config/common_base sed -i s/\(CONFIG_RTE_LIBRTE.*FSLMC.*\)y/\1n/g config/common_base sed -i s/\(CONFIG_RTE_LIBRTE.*OCTEONTX.*\)y/\1n/g config/common_base sed -i s/\(CONFIG_RTE_LIBRTE.*OTX2.*\)y/\1n/g config/common_base sed -i s/\(CONFIG_RTE_LIBRTE.*THUNDERX.*\)y/\1n/g config/common_base # 3. 修复链接器架构问题 sed -i s/CPU_LDFLAGS ?$/CPU_LDFLAGS ? -m elf_x86_64/ mk/arch/x86_64/rte.vars.mk # 4. 安装 64 位 numa 库 sudo dpkg --add-architecture amd64 sudo apt-get update sudo apt-get install -y libnuma-dev:amd64 libnuma1:amd64 # 5. 生成配置 make config Tx86_64-native-linuxapp-gcc # 6. 确保 build/.config 中 ARM 驱动被禁用 sed -i s/\(CONFIG_RTE_LIBRTE.*DPAA.*\)y/\1n/g build/.config sed -i s/\(CONFIG_RTE_LIBRTE.*FSLMC.*\)y/\1n/g build/.config sed -i s/\(CONFIG_RTE_LIBRTE.*OCTEONTX.*\)y/\1n/g build/.config sed -i s/\(CONFIG_RTE_LIBRTE.*OTX2.*\)y/\1n/g build/.config # 7. 编译 make -j$(nproc)3.3 为什么博客里没有这个问题可能的原因DPDK 版本差异不同版本的默认配置不同系统环境差异其他环境可能已安装正确依赖编译方式差异可能使用了 meson 而非 makeUbuntu 版本差异不同 Ubuntu 版本的 binutils 默认行为可能不同四、技术细节说明4.1 PMDINFO 链接过程DPDK 编译过程中的 PMDINFO 步骤源文件 (.c) ↓ gcc -c 目标文件 ↓ dpdk-pmdinfogen PMD 信息文件 ↓ gcc -c PMD 目标文件 ↓ ld -r ← 这里需要指定正确的输出格式 合并目标文件ld -r是可重定位链接不会生成最终可执行文件只是合并多个目标文件。4.2 链接器模拟格式ld -V输出的模拟格式列表格式说明elf_i38632 位 x86 ELFelf_x86_6464 位 x86_64 ELFelf32_x86_6432 位兼容的 x86_64当不指定-m参数时链接器使用列表中的第一个格式elf_i386这就是问题所在。4.3 DPDK 配置优先级config/defconfig_x86_64-native-linuxapp-gcc (最高优先级) config/common_base config/common_linux ↓ build/.config (实际使用的配置)某些配置在defconfig文件中被显式设置会覆盖common_base的值。