Enabling LTO causes linker errors

[rtimpe]

Describe the bug

When compiling with the -flto flag, I get the following error

$ /opt/opencilk-3/bin/clang -fopencilk -flto -v fib.c -o fib
clang version 19.1.7
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/opencilk-3/bin
Build config: +assertions
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/13
Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/13
Candidate multilib: .;@m64
Selected multilib: .;@m64
 "/opt/opencilk-3/bin/clang-19" -cc1 -triple x86_64-unknown-linux-gnu -emit-llvm-bc -flto=full -flto-unit -dumpdir fib- -disable-free -clear-ast-before-backend -main-file-name fib.c -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -fdebug-compilation-dir=/ex -v -fcoverage-compilation-dir=/ex -resource-dir /opt/opencilk-3/lib/clang/19 -internal-isystem /opt/opencilk-3/lib/clang/19/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/13/../../../../x86_64-linux-gnu/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -ferror-limit 19 -fopencilk --opencilk-abi-bitcode=/opt/opencilk-3/lib/clang/19/lib/x86_64-unknown-linux-gnu/libopencilk-abi.bc -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf -fcolor-diagnostics -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/fib-8d58f4.o -x c fib.c
clang -cc1 version 19.1.7 based upon LLVM 19.1.7 default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/usr/lib/gcc/x86_64-linux-gnu/13/../../../../x86_64-linux-gnu/include"
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /opt/opencilk-3/lib/clang/19/include
 /usr/local/include
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.
 "/usr/bin/ld" -z relro --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -pie -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o fib /lib/x86_64-linux-gnu/Scrt1.o /lib/x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/13/crtbeginS.o -L/opt/opencilk-3/lib/clang/19/lib/x86_64-unknown-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/13 -L/usr/lib/gcc/x86_64-linux-gnu/13/../../../../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib64 -L/lib -L/usr/lib -plugin /opt/opencilk-3/bin/../lib/LLVMgold.so -plugin-opt=mcpu=x86-64 --plugin-opt=tapir-target=opencilk --plugin-opt=opencilk-abi-bitcode=/opt/opencilk-3/lib/clang/19/lib/x86_64-unknown-linux-gnu/libopencilk-abi.bc /tmp/fib-8d58f4.o -lopencilk-personality-c -lopencilk -L /opt/opencilk-3/lib/clang/19/lib/x86_64-unknown-linux-gnu -rpath /opt/opencilk-3/lib/clang/19/lib/x86_64-unknown-linux-gnu --enable-new-dtags -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-linux-gnu/13/crtendS.o /lib/x86_64-linux-gnu/crtn.o
/usr/bin/ld: /opt/opencilk-3/bin/../lib/LLVMgold.so: error loading plugin: /opt/opencilk-3/bin/../lib/LLVMgold.so: cannot open shared object file: No such file or directory
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Expected behavior

No error

OpenCilk version

• Release version: 3.0
• Distribution method: Docker image

I also see the same error when using the 2.1 release as a tarball

System information

• OS: Arch linux
• CPU: Intel Core i7-7700K

Steps to reproduce (include relevant output)

This happens with any input code - I just used fib.c from the tutorial as an example.

Working example code

Removing the -flto flag seems to be the only way to make it work

Additional comments

Let me know if I can provide any more information.

[neboat]

The OpenCilk Docker image doesn't include support for LLVM's gold plugin for LTO.

The image should include the LLD linker, however, which supports LTO without the gold plugin. Can you use LLD for LTO in your case, like so?

$ /opt/opencilk-3/bin/clang -fopencilk -flto -v fib.c -o fib -fuse-ld=lld

[rtimpe]

I get the following error

$/opt/opencilk-3/bin/clang -fopencilk -flto -v fib.c -o fib -fuse-ld=lld
clang version 19.1.7
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/opencilk-3/bin
Build config: +assertions
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/13
Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/13
Candidate multilib: .;@m64
Selected multilib: .;@m64
 "/opt/opencilk-3/bin/clang-19" -cc1 -triple x86_64-unknown-linux-gnu -emit-llvm-bc -flto=full -flto-unit -dumpdir fib- -disable-free -clear-ast-before-backend -main-file-name fib.c -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -fdebug-compilation-dir=/ex -v -fcoverage-compilation-dir=/ex -resource-dir /opt/opencilk-3/lib/clang/19 -internal-isystem /opt/opencilk-3/lib/clang/19/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/13/../../../../x86_64-linux-gnu/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -ferror-limit 19 -fopencilk --opencilk-abi-bitcode=/opt/opencilk-3/lib/clang/19/lib/x86_64-unknown-linux-gnu/libopencilk-abi.bc -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf -fcolor-diagnostics -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/fib-69f1c9.o -x c fib.c
clang -cc1 version 19.1.7 based upon LLVM 19.1.7 default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/usr/lib/gcc/x86_64-linux-gnu/13/../../../../x86_64-linux-gnu/include"
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /opt/opencilk-3/lib/clang/19/include
 /usr/local/include
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.
 "/usr/bin/ld.lld" -z relro --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -pie -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o fib /lib/x86_64-linux-gnu/Scrt1.o /lib/x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/13/crtbeginS.o -L/opt/opencilk-3/lib/clang/19/lib/x86_64-unknown-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/13 -L/usr/lib/gcc/x86_64-linux-gnu/13/../../../../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib64 -L/lib -L/usr/lib -plugin-opt=mcpu=x86-64 --plugin-opt=tapir-target=opencilk --plugin-opt=opencilk-abi-bitcode=/opt/opencilk-3/lib/clang/19/lib/x86_64-unknown-linux-gnu/libopencilk-abi.bc /tmp/fib-69f1c9.o -lopencilk-personality-c -lopencilk -L /opt/opencilk-3/lib/clang/19/lib/x86_64-unknown-linux-gnu -rpath /opt/opencilk-3/lib/clang/19/lib/x86_64-unknown-linux-gnu --enable-new-dtags -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-linux-gnu/13/crtendS.o /lib/x86_64-linux-gnu/crtn.o
ld.lld: error: --plugin-opt=: unknown plugin option 'tapir-target=opencilk'
ld.lld: error: --plugin-opt=: unknown plugin option 'opencilk-abi-bitcode=/opt/opencilk-3/lib/clang/19/lib/x86_64-unknown-linux-gnu/libopencilk-abi.bc'
clang: error: linker command failed with exit code 1 (use -v to see invocation)

That being said, I'd actually prefer to not use docker if possible. I submitted the bug report for docker since I was getting the same error with a source build, but when I add the -fuse-ld=lld option to that build, I get

clang-16: error: invalid linker name in argument '-fuse-ld=lld'
clang-16: error: invalid linker name in argument '-fuse-ld=lld'

Do I need to explicitly build lld as part of the opencilk build?

[neboat]

Yes, you need to build lld explicitly as part of the OpenCilk build.

If you're using the infrastructure build script to build your own copy, just add lld to the semicolon-separated list here: https://github.com/OpenCilk/infrastructure/blob/release/tools/build#L75. The revised line in the build script should look something like this:

OPENCILK_COMPONENTS="clang;clang-tools-extra;lld"      # Required components; edit to add extra LLVM projects

Once you modify that line, run the build script. The resulting OpenCilk build should include lld in the bin subdirectory.

Sorry about the Docker image. I think I found the issue with how the Docker image was created, which meant, among other things, that lld was not included. I'll recreate it and upload a new version, for anyone who prefers to use Docker.

Please let me know if you run into any more issues.

[rtimpe]

That seems to have fixed it, thanks!