{2023.06}[NVHPC/25.1-CUDA-12.6] add hook for nvhpc

[adammccartney]

This adds a pre_configure_hook for NVHPC. It performs some search and replace operations on the "localrc" file used by NVHPC to detect information about the system. In particular it points the sysroot flag at the eessi eprefix variable, and appends two variables definitions about where to look for system libraries.

The content of the hook is extracted from:
https://github.com/ComputeCanada/easybuild-computecanada-config/blob/main/2023/cc_hooks.py#L544-L547

[eessi-bot-deucalion]

Instance eessi-bot-deucalion is configured to build for:

• architectures: aarch64/a64fx
• repositories: eessi.io-2023.06-software

[eessi-bot]

Instance eessi-bot-mc-azure is configured to build for:

• architectures: x86_64/amd/zen4
• repositories: eessi.io-2023.06-compat, eessi.io-2023.06-software

[gpu-bot-ugent]

Instance eessi-bot-vsc-ugent is configured to build for:

• architectures: x86_64/amd/zen3
• repositories: eessi-hpc.org-2023.06-software, eessi.io-2023.06-compat, eessi-hpc.org-2023.06-compat, eessi.io-2023.06-software

[eessi-bot-toprichard]

Instance rt-Grace-jr is configured to build for:

• architectures: aarch64/nvidia/grace
• repositories: eessi.io-2023.06-software

[eessi-bot-surf]

Instance eessi-bot-surf is configured to build for:

• architectures: x86_64/amd/zen4, x86_64/amd/zen2
• repositories: eessi-hpc.org-2023.06-software, eessi.io-2023.06-software, eessi.io-2023.06-compat, eessi-hpc.org-2023.06-compat

[eessi-bot]

Instance eessi-bot-mc-aws is configured to build for:

• architectures: x86_64/generic, x86_64/intel/haswell, x86_64/intel/sapphirerapids, x86_64/intel/skylake_avx512, x86_64/intel/cascadelake, x86_64/intel/icelake, x86_64/amd/zen2, x86_64/amd/zen3, aarch64/generic, aarch64/neoverse_n1, aarch64/neoverse_v1
• repositories: eessi.io-2023.06-compat, eessi.io-2023.06-software

[hvelab]

Hi @adammccartney ,

As in the PR there is no build generate with its corresponding tests, could you kindly share the steps you did to test this so we can also test and reproduce it?

Thank you!

[ocaisa]

This is good for now, but there is quite a bit of discussion currently about changing the naming in the EasyBuild context. We are currently naming the compilers only NVHPC but we should perhaps be defining a toolchain hierarchy for NVHPC since they also contain MPI and some math libraries. This may lead to the compilers being called something like nvidia_compilers or NVHPC becoming a fatter toolchain.

[ocaisa]

You're also going to need some versioning clauses here for things you have actually tested.

This is definitely a corner case at present as EESSI itself will not ship his toolchain (currently), and I wonder if we should not allow an environment variable to force the use of the hook, something like

Suggested change

	if self.name == "NVHPC":
	if self.name == "NVHPC":
	force_nvhpc_hook = 'EESSI_FORCE_NVHPC_HOOK'
	if self.version in [...] or os.getenv(force_nvhpc_hook, False):
	...
	else:
	print_msg(f"Not using existing hook for {self.name}/{self.version}, if you wish to force this please set the envvar {force_nvhpc_hook}")

[adammccartney]

Hi @adammccartney ,

As in the PR there is no build generate with its corresponding tests, could you kindly share the steps you did to test this so we can also test and reproduce it?

Thank you!

Sure, would be happy to. Would you mind giving a few points of guidance? Let me know what would be useful to see.
I haven't had the time to look into ReFrame at all yet, so there are no tests (yet) apart from the sanity checks in the EasyBuild. The build last week was done in the eessi-container from the "software layer" repo, which was slightly adapted to suit our own build environment. The build command looks like the following:

#!/bin/bash

project_root="$(realpath $(dirname $(dirname $(dirname $(dirname $BASH_SOURCE)))))"

eb "${project_root}/easyconfigs/2025/NVHPC-25.1-CUDA-12.6.0.eb" \
    -r --cuda-compute-capabilities=9.0 \
    --configfiles="${project_root}/easybuild-asc-config/2025/config.cfg" \
    --hooks="${project_root}/easybuild-asc-config/2025/eb_hooks.py"

As you can see we are referencing an explicit config for easybuild and I think the easyconfig for NVHPC is slightly adapted to include the "accept-eula" variable or whatver. I'll backport this to a "vanilla" eessi-extend environment today that can be used to install stuff on host-injections. I guess it would be useful to have a command that can be run in the standard container started by eessi_container.sh ?

[ocaisa]

Thinking about this a bit more, the logic here could be added directly to the relevant section of the NVHPC easyblock (and used conditionally based on whether the EB build option --sysroot is set).

[ocaisa]

(EPREFIX and sysroot are the same)

[adammccartney]

So, interestingly the sanity check now fails if I try to build this directly on a compute node (x86_64/amd/zen4 is the architecture by the way) . The initial build was done in the eessi container as I mentioned, set up to use the EESSI_PROJECT_INSTALL variable pointing at a writeable /cvmfs/software.asc.ac.at directory.
The build now fails when I try to use EESSI_SITE_INSTALL. Maybe there is something leaking in via the ld cache on the host as was previously observed. Makes me wonder about how usable the compiler is if we load it from the custom cvmfs repo...

[adammccartney]

bot: build inst:eessi-bot-mc-azure arch:x86_64/amd/zen4 repo:eessi.io-2023.06-software accelerator:nvidia/cc90

[eessi-bot]

Updates by the bot instance eessi-bot-mc-aws (click for details)

• account adammccartney has NO permission to send commands to the bot

[eessi-bot]

Updates by the bot instance eessi-bot-mc-azure (click for details)

• account adammccartney has NO permission to send commands to the bot

[eessi-bot-surf]

Updates by the bot instance eessi-bot-surf (click for details)

• account adammccartney has NO permission to send commands to the bot

[gpu-bot-ugent]

Updates by the bot instance eessi-bot-vsc-ugent (click for details)

• account adammccartney has NO permission to send commands to the bot

[adammccartney]

Okay, so I got this working with some careful attention to what the linker was up to.
See the commit message a2fe8be
For a bit more info.