Skip to content

add EESSI version 2025.06 to CI workflows #40

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

add EESSI version 2025.06 to CI workflows #40

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
ea735cf
add 2025.06
bedroge Jul 18, 2025
698d6a5
check if EESSI_VERSION is set to a valid EESSI version
bedroge Jul 22, 2025
0511d6f
check for EasyBuild instead of Python in the given EESSI version
bedroge Jul 22, 2025
1373095
assume EXPECTED_EASYBUILD_VERSION is set
bedroge Jul 22, 2025
22fdc51
define expected EB versions per EESSI version
bedroge Jul 22, 2025
d3d18fe
export EESSI_VERSION and EXPECTED_EASYBUILD_VERSION
bedroge Jul 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 20 additions & 10 deletions .github/workflows/scripts/test_init_scripts.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,16 @@
#!/bin/bash
EESSI_VERSION="2023.06"
export LMOD_PAGER=cat

if [ -z ${EESSI_VERSION} ] || [ ! -d /cvmfs/software.eessi.io/versions/${EESSI_VERSION} ]; then
echo "\$EESSI_VERSION has to be set to a valid EESSI version."
exit 1
fi

if [ -z ${EXPECTED_EASYBUILD_VERSION} ]; then
echo "\$EXPECTED_EASYBUILD_VERSION has to be set to an EasyBuild version that is expected to be available in EESSI version ${EESSI_VERSION}."
exit 1
fi

# initialize assert framework
if [ ! -d assert.sh ]; then
echo "assert.sh not cloned."
Expand Down Expand Up @@ -30,15 +39,16 @@ for shell in ${SHELLS[@]}; do
assert_raises 'echo "${MODULE_SECTIONS[1]}" | grep -E "$PATTERN"'
# TEST 3: Check if module overviews second section is the EESSI init module
assert "echo ${MODULE_SECTIONS[4]}" "/cvmfs/software.eessi.io/versions/$EESSI_VERSION/init/modules"
# Test 4: Load Python module and check version
command="$shell -c 'source init/lmod/$shell 2>/dev/null; module load Python/3.10.8-GCCcore-12.2.0; python --version'"
expected="Python 3.10.8"
assert "$command" "$expected"
# Test 5: Load Python module and check path
PYTHON_PATH=$($shell -c "source init/lmod/$shell 2>/dev/null; module load Python/3.10.8-GCCcore-12.2.0; which python")
PATTERN="/cvmfs/software\.eessi\.io/versions/$EESSI_VERSION/software/linux/x86_64/(intel/haswell|amd/zen3)/software/Python/3\.10\.8-GCCcore-12\.2\.0/bin/python"
echo "$PYTHON_PATH" | grep -E "$PATTERN"
assert_raises 'echo "$PYTHON_PATH" | grep -E "$PATTERN"'
# Test 4: Load EasyBuild module and check version
# eb --version outputs: "This is EasyBuild 5.1.1 (framework: 5.1.1, easyblocks: 5.1.1) on host ..."
command="$shell -c 'source init/lmod/$shell 2>/dev/null; module load EasyBuild/${EXPECTED_EASYBUILD_VERSION}; eb --version | cut -d \" \" -f4'"
assert "$command" "$EXPECTED_EASYBUILD_VERSION"
# Test 5: Load EasyBuild module and check path
EASYBUILD_PATH=$($shell -c "source init/lmod/$shell 2>/dev/null; module load EasyBuild/${EXPECTED_EASYBUILD_VERSION}; which eb")
# escape the dots in ${EASYBUILD_VERSION}
PATTERN="/cvmfs/software\.eessi\.io/versions/$EESSI_VERSION/software/linux/x86_64/(intel/haswell|amd/zen3)/software/EasyBuild/${EXPECTED_EASYBUILD_VERSION//./\\.}/bin/eb"
echo "$EASYBUILD_PATH" | grep -E "$PATTERN"
assert_raises 'echo "$EASYBUILD_PATH" | grep -E "$PATTERN"'

#End Test Suite
assert_end "source_eessi_$shell"
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/test-eb-hooks.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ jobs:
matrix:
EESSI_VERSION:
- '2023.06'
- '2025.06'
steps:
- name: Check out software-layer repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/tests_archdetect_nvidia_gpu.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ jobs:
- cc01 # non-existing GPU
EESSI_VERSION:
- '2023.06'
- '2025.06'
fail-fast: false
steps:
- name: checkout
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/tests_eessi_extend_module.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,11 @@ jobs:
matrix:
EESSI_VERSION:
- '2023.06'
- '2025.06'
steps:
- name: Check out software-layer repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1

- name: Mount EESSI CernVM-FS repository
uses: eessi/github-action-eessi@v3
with:
Expand Down
11 changes: 7 additions & 4 deletions .github/workflows/tests_eessi_module.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,11 @@ jobs:
matrix:
EESSI_VERSION:
- '2023.06'
- '2025.06'
steps:
- name: Check out software-layer repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1

- name: Mount EESSI CernVM-FS pilot repository
uses: cvmfs-contrib/github-action-cvmfs@55899ca74cf78ab874bdf47f5a804e47c198743c # v4.0
with:
Expand Down Expand Up @@ -69,6 +70,7 @@ jobs:
matrix:
EESSI_VERSION:
- '2023.06'
- '2025.06'
EESSI_SOFTWARE_SUBDIR_OVERRIDE:
- x86_64/amd/zen3
- x86_64/amd/zen4
Expand All @@ -80,7 +82,7 @@ jobs:
steps:
- name: Check out software-layer repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1

- name: Mount EESSI CernVM-FS pilot repository
uses: cvmfs-contrib/github-action-cvmfs@55899ca74cf78ab874bdf47f5a804e47c198743c # v4.0
with:
Expand Down Expand Up @@ -141,9 +143,9 @@ jobs:
echo ""
if (diff "${moduleoutfile}" "${sourceoutfile}" > /dev/null); then
echo "Test for checking env variables PASSED"
else
else
echo "Test for checking env variables FAILED" >&2
diff --unified=0 "${moduleoutfile}" "${sourceoutfile}"
diff --unified=0 "${moduleoutfile}" "${sourceoutfile}"
exit 1
fi

Expand All @@ -154,6 +156,7 @@ jobs:
matrix:
EESSI_VERSION:
- '2023.06'
- '2025.06'
EESSI_SOFTWARE_SUBDIR_OVERRIDE:
- none
- x86_64/amd/zen2
Expand Down
10 changes: 9 additions & 1 deletion .github/workflows/tests_init_module.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,14 @@ jobs:
matrix:
EESSI_VERSION:
- '2023.06'
- '2025.06'
EESSI_SOFTWARE_SUBDIR_OVERRIDE:
- x86_64/intel/haswell
include:
- EESSI_VERSION: '2023.06'
EXPECTED_EASYBUILD_VERSION: '5.0.0'
- EESSI_VERSION: '2025.06'
EXPECTED_EASYBUILD_VERSION: '5.1.1'
steps:
- name: Check out software-layer repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
Expand All @@ -33,7 +39,7 @@ jobs:
for shell in $(ls init/lmod); do
sed -i "s/__EESSI_VERSION_DEFAULT__/${{matrix.EESSI_VERSION}}/g" init/lmod/${shell}
done

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that .github/workflows/scripts/test_init_scripts.sh is currently version specific (causing this CI to fail)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apart from this, the rest of the CI is set up so that it will only pass once the accelerator directories actually exist. To get it it to pass we could manually create the structure?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or we build a relevant CUDA first and then fix the CI

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can easily remove that hardcoded EESSI_VERSION in .github/workflows/scripts/test_init_scripts.sh. However, it's also loading a particular module (https://github.com/EESSI/software-layer-scripts/blob/main/.github/workflows/scripts/test_init_scripts.sh#L34), how do we deal with that?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can have it load EasyBuild/5.1.1 and use eb --version instead, that specific version appears in both.

You could also use an environment variable that might be EESSI version specific and use that, I did that in https://github.com/EESSI/software-layer-scripts/blob/main/.github/workflows/scripts/verify_eessi_environment.py#L49 (via https://github.com/EESSI/software-layer-scripts/blob/main/.github/workflows/tests_eessi_module.yml#L196)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That CI workflow is now checking for an EB version (which can be specified in the workflow matrix), and this seems to work. For the accelerators I guess we should indeed just do a CUDA build soon.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest we do this piece by piece, fixing the things we can first.

For the specific case of accelerators, we can enable it once the directory exists. I wonder if there are different ways to fail? Can we report an "unable to run" if a comparison directory does not exist?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems it can at least be made more visual:

echo "::warning:: Forced to skip check, comparison directory {} does not exist"

- name: Clone assert.sh script
run: git clone https://github.com/lehmannro/assert.sh.git

Expand All @@ -45,5 +51,7 @@ jobs:

- name: Run tests for available shells
run: |
export EESSI_VERSION=${{matrix.EESSI_VERSION}}
export EXPECTED_EASYBUILD_VERSION=${{matrix.EXPECTED_EASYBUILD_VERSION}}
.github/workflows/scripts/test_init_scripts.sh "bash" "zsh" "ksh" "fish" "csh"

1 change: 1 addition & 0 deletions .github/workflows/tests_scripts.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ jobs:
matrix:
EESSI_VERSION:
- '2023.06'
- '2025.06'
steps:
- name: checkout
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
Expand Down
Loading