Skip to content

Add new GPU workflow #949

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
87 changes: 87 additions & 0 deletions .github/workflows/test-check_easystacks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# documentation: https://help.github.com/en/articles/workflow-syntax-for-github-actions
name: Check new and changed easystacks
on:
push:
branches: [ "*-software.eessi.io" ]
pull_request:
workflow_dispatch:
permissions:
contents: read # to fetch code (actions/checkout)
env:
EESSI_ACCELERATOR_TARGETS: |
x86_64/amd/zen2:
- nvidia/cc80
x86_64/amd/zen3:
- nvidia/cc80
jobs:
check_missing:
runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
EESSI_VERSION:
- 2023.06
EESSI_SOFTWARE_SUBDIR_OVERRIDE:
- x86_64/amd/zen2
- x86_64/amd/zen3
- x86_64/amd/zen4
- x86_64/intel/haswell
- x86_64/intel/skylake_avx512
- x86_64/generic
steps:
- name: Check out software-layer repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
fetch-depth: 0
- name: Check for changed easystackfiles in pr
run: |
echo "GITHUB_BASE_REF = ${GITHUB_BASE_REF}"
echo $GITHUB_ENV
echo "git diff --no-renames --name-status ${GITHUB_BASE_REF}..HEAD"
export CHANGED=$(git diff --no-renames --name-status origin/${GITHUB_BASE_REF}..HEAD)
echo $CHANGED
python software-layer-scripts/.github/workflows/scripts/filter_git_diff.py
- name: Mount EESSI CernVM-FS pilot repository
if: env.CHANGED_EASYSTACKS
uses: cvmfs-contrib/github-action-cvmfs@55899ca74cf78ab874bdf47f5a804e47c198743c # v4.0
with:
cvmfs_config_package: https://github.com/EESSI/filesystem-layer/releases/download/latest/cvmfs-config-eessi_latest_all.deb
cvmfs_http_proxy: DIRECT
cvmfs_repositories: software.eessi.io
- name: Test check_missing_installations.sh script
if: env.CHANGED_EASYSTACKS
run: |
export EESSI_SOFTWARE_SUBDIR_OVERRIDE=${{matrix.EESSI_SOFTWARE_SUBDIR_OVERRIDE}}
source /cvmfs/software.eessi.io/versions/${{matrix.EESSI_VERSION}}/init/bash
# set $EESSI_CPU_FAMILY to the CPU architecture that corresponds to $EESSI_SOFTWARE_SUBDIR_OVERRIDE (part before the first slash),
# to prevent issues with checks in the Easybuild configuration that use this variable
export EESSI_CPU_FAMILY=${EESSI_SOFTWARE_SUBDIR_OVERRIDE%%/*}
module load EasyBuild
which eb
eb --version
export EESSI_PREFIX=/cvmfs/software.eessi.io/versions/${{matrix.EESSI_VERSION}}
export EESSI_OS_TYPE=linux
env | grep ^EESSI | sort

# check the changed easystack files
echo "check for missing installation in changed easystack files"
easystack_files=(${CHANGED_EASYSTACKS})

# check for missing installations if a file in the easystacks directory is changed
# check easystack file for the use of `--from-pr`
for easystack_file in ${easystack_files[@]}; do
echo "check for the use of from-pr in ${easystack_file}"
if grep -q "from-pr" ${easystack_file}; then
echo "ERROR: from-pr is found in ${easystack_file} please use from-commit" >&2
exit 1
fi

echo "check missing installatios for ${easystack_file}..."
./check_missing_installations.sh ${easystack_file} || ec=$?
export eb_missing_out=$(./check_missing_installations.sh ${easystack_file})
if [[ ${ec} -ne 0 ]]; then
echo "missing installations found for ${easystack_file}!" >&2
export missing=$(env | grep .eb\) | tr -d \*)
python software-layer-scripts/.github/workflows/scripts/parse_missing-installations-output.py
fi
done
8 changes: 4 additions & 4 deletions .github/workflows/test-software.eessi.io.yml
Original file line number Diff line number Diff line change
Expand Up @@ -94,8 +94,8 @@ jobs:
module load EasyBuild/${eb_version}
which eb
eb --version
software-layer-scripts/check_missing_installations.sh ${easystack_file}
ec=$?
ec=0
software-layer-scripts/check_missing_installations.sh ${easystack_file} || ec=$?
if [[ ${ec} -ne 0 ]]; then echo "missing installations found for ${easystack_file}!" >&2; exit ${ec}; fi
done

Expand All @@ -114,8 +114,8 @@ jobs:
module load EasyBuild/${eb_version}
which eb
eb --version
software-layer-scripts/check_missing_installations.sh ${easystack_file}
ec=$?
ec=0
software-layer-scripts/check_missing_installations.sh ${easystack_file} || ec=$?
if [[ ${ec} -ne 0 ]]; then echo "missing installations found for ${easystack_file}!" >&2; exit ${ec}; fi
done
module unuse ${EESSI_SOFTWARE_PATH}/accel/${accel}/modules/all
Expand Down
Loading