Packaging OpenCilk for Nix

[Noir01]

Hi OpenCilk community,

I've recently been working on Gibbon which uses Cilk and so is confined to GCC7. As part of a small effort to modernize its dependencies, I've been looking into OpenCilk which appears to be a drop-in replacement. However, it not yet being packaged for Nix is hindering these efforts, and so I've been working on packaging it myself. I've run into some roadblocks however, and I'd like to ask if anyone familiar with building OpenCilk from source would want to take a look and maybe offer some insight.

The Nix derivation file is here opencilk.nix along with its crashlog.txt. I'd really appreciate any help regarding this!

Thank you!

[neboat]

Hey @Noir01,

Thanks for reaching out! We definitely have expertise on building OpenCilk from source, and we would be happy to help.

Thanks for sharing the log and Nix derivation file. We'll take a look and get back to you soon.

[VoxSciurorum]

I see an error "Please use architecture with 4 or 8 byte pointers." It looks like cmake is confused about your environment. This can be caused by the compiler crashing or by cmake files that don't understand your OS. The first step should be to upgrade your OpenCilk from version 2.1 to 3.0. We released version 3.0 in May.

https://github.com/OpenCilk/opencilk-project/releases

If you're stuck on 2.1 for some reason let us know. We can try to set up a NixOS VM to test.

[Noir01]

I tried a trivial switch from v2.1 to v3.0 8e2fd46, but it resulted in a very similar crash.

I also noticed the error you pointed out, but I don't think that is the root issue. Rather, I believe the issue arises when building runtimes. Take a look at these logs:

[100%] No download step for 'runtimes'�
[100%] No update step for 'runtimes'�
[100%] No patch step for 'runtimes'�
[100%] Performing configure step for 'runtimes'�
Not searching for unused variables given on the command line.
CMake Deprecation Warning at /nix/store/p26ilwf70haa7sxlkdfas5qrs2sx1zrf-combine-all-repos/infrastructure/opencilk/cmake/Modules/CMakePolicy.cmake:11 (cmake_policy):
  The OLD behavior for policy CMP0114 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.
Call Stack (most recent call first):
  CMakeLists.txt:6 (include)

CMake Deprecation Warning at /nix/store/p26ilwf70haa7sxlkdfas5qrs2sx1zrf-combine-all-repos/infrastructure/opencilk/cmake/Modules/CMakePolicy.cmake:16 (cmake_policy):
  The OLD behavior for policy CMP0116 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.
Call Stack (most recent call first):
  CMakeLists.txt:6 (include)

-- The C compiler identification is Clang 19.1.7
-- The CXX compiler identification is Clang 19.1.7
-- The ASM compiler identification is Clang with GNU-like command-line
-- Found assembler: /build/combine-all-repos/build/bin/clang
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - failed
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - failed
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for histedit.h
-- Looking for histedit.h - not found
-- Could NOT find LibEdit (missing: LibEdit_INCLUDE_DIRS LibEdit_LIBRARIES) 
-- Found ZLIB: /nix/store/lv6nackqis28gg7l2ic43f6nk52hb39g-zlib-1.3.1/lib/libz.so (found version "1.3.1")
-- Found LibXml2: /nix/store/kf7fzax0av9him9m4zcmbqmihh5108wq-libxml2-2.12.7/lib/libxml2.so (found version "2.12.7")
-- Could NOT find LibEdit (missing: LibEdit_INCLUDE_DIRS LibEdit_LIBRARIES) 
-- Performing Test CXX_SUPPORTS_UNWINDLIB_EQ_NONE_FLAG
-- Performing Test CXX_SUPPORTS_UNWINDLIB_EQ_NONE_FLAG - Failed
-- Performing Test CXX_SUPPORTS_NOSTDLIBXX_FLAG
-- Performing Test CXX_SUPPORTS_NOSTDLIBXX_FLAG - Failed
-- Performing Test CXX_SUPPORTS_NOSTDINCXX_FLAG
-- Performing Test CXX_SUPPORTS_NOSTDINCXX_FLAG - Failed
......

I think for some reason when runtimes are starting their build process, the compiler crashes completely.

[neboat]

There should be CMake logs specifically for runtimes that might be useful here. I think those logs should be in a runtimes-specific subdirectory of your build directory, e.g., build/runtimes/runtimes-bins/CMakeFiles. Can you share the CMakeErrors.log and CMakeConfigureLog.yaml in that directory?

[Noir01]

I didn't find any CMakeErrors.log, but I did find a /build/runtimes/runtimes-bins/CMakeFiles/CMakeConfigureLog.yaml. Here is the file.

[neboat]

Thanks for sharing the CMakeConfigureLog.yaml.

You can see from that log that nearly all of CMake's checks fail with errors similar to this:

/nix/store/x7w0rmkaam7aikclyhwdbn8wmmq29ma6-binutils-2.44/bin/ld: cannot find Scrt1.o: No such file or directory
/nix/store/x7w0rmkaam7aikclyhwdbn8wmmq29ma6-binutils-2.44/bin/ld: cannot find crti.o: No such file or directory
/nix/store/x7w0rmkaam7aikclyhwdbn8wmmq29ma6-binutils-2.44/bin/ld: cannot find crtbeginS.o: No such file or directory
/nix/store/x7w0rmkaam7aikclyhwdbn8wmmq29ma6-binutils-2.44/bin/ld: cannot find -lgcc: No such file or directory

These errors indicate that, when CMake tries to compile and link something, the link step fails because it cannot find these core C-runtime object files and libraries.

I don't know details about Nix packaging, but it looks like similar issues come up when building LLVM/clang from source in Nix and that these issues can be handled by setting certain environment variables appropriately: https://wiki.nixos.org/wiki/LLVM. Building OpenCilk from source is very similar to building LLVM/clang from source, because OpenCilk's build systems is based on LLVM's. See if you can use similar environment variables to fix your OpenCilk build.

[Noir01]

I tried some things over the past week to try and get it to build but to no avail. First, I tried inserting the environment variables mentioned in https://wiki.nixos.org/wiki/LLVM, but that did not seem to fix stuff. Then I started looking into how wrappers and how Nixpkgs deal with official LLVM, and found this https://github.com/NixOS/nixpkgs/blob/25.05/pkgs/build-support/cc-wrapper/default.nix. It seemed exactly like what we need, and so I tried replicating how Nixpkgs does it, but to no avail and with an identical error as far as I understand. Here is the relevant commit.

Right now, I'm wondering if the build process for the runtimes makes any kind of assumptions about the environment that aren't Nix-compatible. I don’t have much experience with CMake or projects of this scale, so I’d really appreciate any insight on how the runtimes are built, or tips on other ways to move forward.

[neboat]

The unusual thing about the runtimes LLVM subcomponent is that it uses the just-built clang binary — built as a result of -DLLVM_ENABLE_PROJECTS=clang — to build the specified runtimes. As such, it relies on that just-built clang binary to find system libraries and header files.

The Nix LLVM package seems to use the same runtimes mechanism OpenCilk uses, based on this line: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/compilers/llvm/common/libcxx/default.nix#L114. OpenCilk doesn't use the runtimes mechanism in any unusual way, which makes me think there should be a way to make this work in Nix. But I can't tell how the LLVM Nix package handles the issues you're seeing. Maybe the maintainers of the LLVM Nix package could offer guidance?

I noticed that the shell.nix on https://wiki.nixos.org/wiki/LLVM specifies -DGCC_INSTALL_PREFIX=${gcc} and -DC_INCLUDE_DIRS=${stdenv.cc.libc.dev}/include among the CMake flags. Have you tried setting those flags for building OpenCilk? I know the LLVM community deprecated -DGCC_INSTALL_PREFIX recently, and it looks like you're using the -DCLANG_CONFIG_FILE_SYSTEM_DIR CMake option instead. But I can't see what's in the $out/etc/clang file to tell whether that file specifies the correct --gcc-install-dir= option, discussed here: https://discourse.llvm.org/t/add-gcc-install-dir-deprecate-gcc-toolchain-and-remove-gcc-install-prefix/65091.

Another option might be to build cheetah and cilktools as separate Nix packages that depend on the OpenCilk compiler, rather than build them as part of the OpenCilk package. That way, you can remove the -DLLVM_ENABLE_RUNTIMES CMake option from the OpenCilk build and avoid these issues with the runtimes build. We have some support for building cheetah and cilktools as stand-alone repositories, as described in the README's on those GitHub repos. For some examples, we build cheetah and cilktools as stand-alone components in OpenCilk's GitHub actions: https://github.com/OpenCilk/actions/blob/main/build-cheetah/action.yml#L57-L70, https://github.com/OpenCilk/actions/blob/main/build-cilktools/action.yml#L57-L69. I'm not sure if that process is compatible with Nix packaging, however. (For example, building cheetah and cilktools as separate standalone packages isn't currently compatible with Homebrew packaging.)