Nvidia driver fails to build on Linux kernels with CONFIG_DEBUG_INFO_BTF_MODULES

[ryao]

NVIDIA Open GPU Kernel Modules Version

565.77

Operating System and Version

Gentoo Linux

Kernel Release

6.12.6-gentoo-x86_64

Please confirm you are running a stable release kernel (e.g. not a -rc). We do not accept bug reports for unreleased kernels.

• I am running on a stable kernel release.

Build Command

emerge x11-drivers/nvidia-drivers

Terminal output/Build Log

build.log

More Info

That build used MAKEOPTS=-j16, so the command that caused the failure is not obvious. The command that fails is this:

# LLVM_OBJCOPY="/usr/bin/x86_64-pc-linux-gnu-objcopy" pahole -J -j --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func,decl_tag_kfuncs --lang_exclude=rust --btf_features=distilled_base --btf_base vmlinux /var/tmp/portage/x11-drivers/nvidia-drivers-565.77/work/kernel-module-source/kernel-open/nvidia-modeset.ko
dwarf_expr: unhandled 0x12 DW_OP_ operation
die__process_function: tag not supported 0x2f (template_type_parameter)!
dwarf_expr: unhandled 0x12 DW_OP_ operation
Unsupported DW_TAG_reference_type(0x10): type: 0x5fb
Encountered error while encoding BTF.

This appears to be related to #149. The reporter there claimed using !buildflags on Arch avoided the problem, although I am not sure how that is possible because the kernel build system does not use regular build flags.

In any case, there is something in the module's DWARF information that pahole does not like. Upgrading to pahole 1.28, which is the latest version, does not resolve the issue.

[ryao]

Renaming my /usr/src/linux/vmlinux binary will allow the build to complete, but this is a hack, as it prevents the BTF debuginfo from being generated.

[ryao]

It probably should be noted that reproduction likely requires building against a locally built kernel. I assume that a packaged one won't include the vmlinux binary in /usr/src/linux.

[Binary-Eater]

I made an internal fix for this that has not made its way into a public release. With that fix, you will not need to move vmlinux, and BTF will generate.

[Binary-Eater]

Here is a commit with our internal fix for this issue: Binary-Eater@854449a

[arnav-kansal]

Upgrading to pahole 1.28, which is the latest version, does not resolve the issue.

This issue is faced only on newer pahole versions. Older pahole versions do not generate BTF for these modules at all, so an update is unlikely to solve.

As the reporter is using a locally built kernel they can skip BTF generation in the meanwhile as well

diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
--- a/scripts/Makefile.modfinal
+++ b/scripts/Makefile.modfinal
@@ -41,6 +41,8 @@
       cmd_btf_ko = 							\
 	if [ ! -f vmlinux ]; then					\
 		printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
+	elif echo $@ | grep -q "nvidia"; then \
+		printf "Skipping BTF generation for %s because it's an Nvidia module (C++)\n" $@ 1>&2; \
 	else								\
 		LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) --btf_base vmlinux $@; \
 		$(RESOLVE_BTFIDS) -b vmlinux $@; 			\

[ryao]

Here is a commit with our internal fix for this issue: Binary-Eater@854449a

Thanks again for this. Unfortunately, I did not get around to applying this today when I updated to 570.86.16. Without this patch, the build process will hang. With this patch, the build process will also hang. It also does not apply cleanly to 565.77. This issue made troubleshooting #773 "interesting".