Skip to content

Commit

Permalink
ci: Fix broken CI for ASWF 2021 and 2022 containers
Browse files Browse the repository at this point in the history
The background here takes some explaining...

The ASWF's prepared Docker containers that reproduce a CentOS7-based
environment corresponding to VFX Platform 2021 and 2022 years contain
a glibc that is too old to run a newer version of "node" -- which
GitHub Actions themselves use. The old one is no longer receiving
support, and GitHub finally changed the policy that was allowing its
GHA runners to still run the old version that worked with those ASWF
containers.  The new node version that is happy on the GHA runners
requires a glibc newer than the one in the containers. So that breaks
all our CI runs in 2021 and 2022 VFX Platform configurations.

See this thread, if you dare:
https://academysoftwarefdn.slack.com/archives/C0169RX7MMK/p1732574400981949

Anyway, Jean-Francois Panisset proposed and tested a fix, which lets
us continue working (for now) by downloading an old node implemenation
(which is happy with the old glibc) and installing it atop the
conainers, so it's this old one that the GH actions will use. It feels
like it's all held together by chewing gum and duct tape, but it
unbreaks our CI and hopefully it will keep working until all the
studios have switched fully to RHEL/Alma/Rocky 9.x and we no longer
care about testing on CentOS-7 based containers or supporting the VFX
Platform years that are too old to be relevant to the new OS that the
studios are using.

Signed-off-by: Larry Gritz <[email protected]>
  • Loading branch information
lgritz committed Nov 29, 2024
1 parent ff0861f commit 7179e2c
Showing 1 changed file with 168 additions and 58 deletions.
226 changes: 168 additions & 58 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,159 @@ permissions: read-all
jobs:

aswf:
name: "VFX${{matrix.vfxyear}} ${{matrix.desc}}"
strategy:
fail-fast: false
matrix:
include:
- desc: gcc11/C++17 py3.10 exr3.1 ocio2.2
nametag: linux-vfx2023
runner: ubuntu-latest
container: aswftesting/ci-osl:2023-clang15
vfxyear: 2023
cxx_std: 17
python_ver: "3.10"
simd: "avx2,f16c"
fmt_ver: 10.1.1
pybind11_ver: v2.10.0
- desc: gcc11/C++17 py3.11 exr3.2 ocio2.3
nametag: linux-vfx2024
runner: ubuntu-latest
container: aswftesting/ci-osl:2024-clang17
vfxyear: 2024
cxx_std: 17
python_ver: "3.11"
simd: "avx2,f16c"
fmt_ver: 10.1.1
pybind11_ver: v2.12.0
- desc: sanitizers
nametag: sanitizer
runner: ubuntu-latest
container: aswf/ci-osl:2024-clang17
vfxyear: 2024
cc_compiler: clang
cxx_compiler: clang++
cxx_std: 17
opencolorio_ver: v2.3.2
python_ver: "3.11"
setenvs: export SANITIZE=address,undefined
OIIO_CMAKE_FLAGS="-DSANITIZE=address,undefined -DUSE_PYTHON=0"
CMAKE_BUILD_TYPE=Debug
CTEST_TEST_TIMEOUT=1200
CTEST_EXCLUSIONS="broken|png-damaged"
# Test ABI stability. `abi_check` is the version or commit that we
# believe is the current standard against which we don't want to
# break the ABI. Basically, we will build that version as well as
# the current one, and compare the resulting libraries.
- desc: abi check
nametag: abi-check
runner: ubuntu-latest
container: aswftesting/ci-osl:2023-clang15
vfxyear: 2023
cxx_std: 17
python_ver: "3.10"
simd: "avx2,f16c"
fmt_ver: 10.1.1
pybind11_ver: v2.10.0
skip_tests: 1
# abi_check: v2.6.2.0
abi_check: 7929c81619ea8201065c475761546d51091f6773
setenvs: export OIIO_CMAKE_FLAGS="-DOIIO_BUILD_TOOLS=0 -DOIIO_BUILD_TESTS=0 -DUSE_PYTHON=0"
USE_OPENCV=0 USE_FFMPEG=0 USE_PYTHON=0 USE_FREETYPE=0
CMAKE_BUILD_TYPE=RelWithDebInfo

runs-on: ${{ matrix.runner }}
container:
image: ${{ matrix.container }}
env:
CXX: ${{matrix.cxx_compiler}}
CC: ${{matrix.cc_compiler}}
CMAKE_CXX_STANDARD: ${{matrix.cxx_std}}
USE_SIMD: ${{matrix.simd}}
FMT_VERSION: ${{matrix.fmt_ver}}
OPENCOLORIO_VERSION: ${{matrix.opencolorio_ver}}
OPENEXR_VERSION: ${{matrix.openexr_ver}}
PYBIND11_VERSION: ${{matrix.pybind11_ver}}
PYTHON_VERSION: ${{matrix.python_ver}}
ABI_CHECK: ${{matrix.abi_check}}
steps:
# We would like to use harden-runner, but it flags too many false
# positives, every time we download a dependency. We should use it only
# on CI runs where we are producing artifacts that users might rely on.
# - name: Harden Runner
# uses: step-security/harden-runner@248ae51c2e8cc9622ecf50685c8bf7150c6e8813 # v1.4.3
# with:
# egress-policy: audit # TODO: change to 'egress-policy: block' after couple of runs
- uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # v3.6.0
# Note: can't upgrade to actions/checkout 4.0 because it needs newer
# glibc than these containers have.
- name: Prepare ccache timestamp
id: ccache_cache_keys
run: echo "date=`date -u +'%Y-%m-%dT%H:%M:%SZ'`" >> $GITHUB_OUTPUT
- name: ccache
id: ccache
uses: actions/cache@e12d46a63a90f2fae62d114769bbf2a179198b5c # v3.3.3
# Note: can't upgrade to actions/cache 4.0 because it needs newer
# glibc than these containers have.
with:
path: /tmp/ccache
key: ${{github.job}}-${{matrix.nametag}}-${{steps.ccache_cache_keys.outputs.date}}
restore-keys: ${{github.job}}-
- name: Build setup
run: |
${{matrix.setenvs}}
src/build-scripts/ci-startup.bash
- name: Dependencies
run: |
${{matrix.depcmds}}
src/build-scripts/gh-installdeps.bash
- name: Build
if: matrix.skip_build != '1'
run: src/build-scripts/ci-build.bash
- name: Testsuite
if: matrix.skip_tests != '1'
run: src/build-scripts/ci-test.bash
- name: Check out ABI standard
if: matrix.abi_check != ''
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
ref: ${{matrix.abi_check}}
path: abi_standard
- name: Build ABI standard
if: matrix.abi_check != ''
run: |
mkdir -p abi_standard/build
pushd abi_standard
src/build-scripts/ci-build.bash
popd
- name: Check ABI
if: matrix.abi_check != ''
run: |
src/build-scripts/ci-abicheck.bash ./build abi_standard/build libOpenImageIO libOpenImageIO_Util
- name: Build Docs
if: matrix.build_docs == '1'
run: |
cd src/doc
time make doxygen
time make sphinx
- uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3
# Note: versions of upload-artifact >= 4.0 can't run on the ASWF
# containers because they need a newer glibc than they have.
if: ${{ failure() || matrix.build_docs == '1'}}
with:
name: oiio-${{github.job}}-${{matrix.nametag}}
path: |
build/cmake-save
build/compat_reports
build/sphinx
build/testsuite/*/*.*
!build/testsuite/oiio-images
!build/testsuite/openexr-images
!build/testsuite/fits-images
!build/testsuite/j2kp4files_v1_5
aswf-old:
name: "VFX${{matrix.vfxyear}} ${{matrix.desc}}"
strategy:
fail-fast: false
Expand All @@ -42,6 +195,7 @@ jobs:
runner: ubuntu-latest
container: aswf/ci-osl:2021
vfxyear: 2021
old_node: 1
cxx_std: 17
openexr_ver: v3.1.13
python_ver: 3.7
Expand All @@ -55,6 +209,7 @@ jobs:
runner: ubuntu-latest
container: aswf/ci-osl:2021-clang10
vfxyear: 2021
old_node: 1
cc_compiler: clang
cxx_compiler: clang++
cxx_std: 17
Expand All @@ -70,6 +225,7 @@ jobs:
runner: ubuntu-latest
container: aswf/ci-osl:2022-clang11
vfxyear: 2022
old_node: 1
cxx_std: 17
python_ver: 3.9
simd: "avx2,f16c"
Expand All @@ -81,6 +237,7 @@ jobs:
runner: ubuntu-latest
container: aswf/ci-osl:2022-clang13
vfxyear: 2022
old_node: 1
cc_compiler: clang
cxx_compiler: clang++
cxx_std: 17
Expand All @@ -94,6 +251,7 @@ jobs:
runner: ubuntu-latest
container: aswf/ci-osl:2022
vfxyear: 2022
old_node: 1
cxx_std: 17
opencolorio_ver: v2.2.1
python_ver: 3.9
Expand All @@ -112,6 +270,7 @@ jobs:
runner: ubuntu-latest
container: aswf/ci-osl:2022
vfxyear: 2022
old_node: 1
cc_compiler: icx
cxx_compiler: icpx
cxx_std: 17
Expand All @@ -127,47 +286,13 @@ jobs:
# to use g++.
# Building libuhdr with icx results in test failures
# so we force using gcc/g++.
- desc: sanitizers
nametag: sanitizer
runner: ubuntu-latest
container: aswf/ci-osl:2024-clang17
vfxyear: 2024
cc_compiler: clang
cxx_compiler: clang++
cxx_std: 17
opencolorio_ver: v2.3.2
python_ver: "3.11"
setenvs: export SANITIZE=address,undefined
OIIO_CMAKE_FLAGS="-DSANITIZE=address,undefined -DUSE_PYTHON=0"
CMAKE_BUILD_TYPE=Debug
CTEST_TEST_TIMEOUT=1200
CTEST_EXCLUSIONS="broken|png-damaged"
- desc: gcc11/C++17 py3.10 exr3.1 ocio2.2
nametag: linux-vfx2023
runner: ubuntu-latest
container: aswftesting/ci-osl:2023-clang15
vfxyear: 2023
cxx_std: 17
python_ver: "3.10"
simd: "avx2,f16c"
fmt_ver: 10.1.1
pybind11_ver: v2.10.0
- desc: gcc11/C++17 py3.11 exr3.2 ocio2.3
nametag: linux-vfx2024
runner: ubuntu-latest
container: aswftesting/ci-osl:2024-clang17
vfxyear: 2024
cxx_std: 17
python_ver: "3.11"
simd: "avx2,f16c"
fmt_ver: 10.1.1
pybind11_ver: v2.12.0
- desc: oldest gcc9.3/C++17 py3.7 exr-3.1
# Oldest versions of the dependencies that we support.
nametag: linux-oldest
runner: ubuntu-latest
container: aswf/ci-osl:2021
vfxyear: 2021
old_node: 1
cxx_std: 17
fmt_ver: 7.0.1
opencolorio_ver: v2.2.1
Expand All @@ -186,6 +311,7 @@ jobs:
runner: ubuntu-latest
container: aswf/ci-osl:2021
vfxyear: 2021
old_node: 1
cxx_std: 17
fmt_ver: 7.0.1
opencolorio_ver: v2.2.1
Expand All @@ -202,30 +328,12 @@ jobs:
FREETYPE_VERSION=VER-2-10-0
depcmds: sudo rm -rf /usr/local/include/OpenEXR

# Test ABI stability. `abi_check` is the version or commit that we
# believe is the current standard against which we don't want to
# break the ABI. Basically, we will build that version as well as
# the current one, and compare the resulting libraries.
- desc: abi check
nametag: abi-check
runner: ubuntu-latest
container: aswftesting/ci-osl:2023-clang15
vfxyear: 2023
cxx_std: 17
python_ver: "3.10"
simd: "avx2,f16c"
fmt_ver: 10.1.1
pybind11_ver: v2.10.0
skip_tests: 1
# abi_check: v2.6.2.0
abi_check: 7929c81619ea8201065c475761546d51091f6773
setenvs: export OIIO_CMAKE_FLAGS="-DOIIO_BUILD_TOOLS=0 -DOIIO_BUILD_TESTS=0 -DUSE_PYTHON=0"
USE_OPENCV=0 USE_FFMPEG=0 USE_PYTHON=0 USE_FREETYPE=0
CMAKE_BUILD_TYPE=RelWithDebInfo

runs-on: ${{ matrix.runner }}
container:
image: ${{ matrix.container }}
volumes:
- /node20217:/node20217:rw,rshared
- /node20217:/__e/node20:ro,rshared
env:
CXX: ${{matrix.cxx_compiler}}
CC: ${{matrix.cc_compiler}}
Expand All @@ -237,9 +345,11 @@ jobs:
PYBIND11_VERSION: ${{matrix.pybind11_ver}}
PYTHON_VERSION: ${{matrix.python_ver}}
ABI_CHECK: ${{matrix.abi_check}}
ACTIONS_RUNNER_FORCE_ACTIONS_NODE_VERSION: node16
ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true
steps:
- name: install nodejs20glibc2.17
if: matrix.old_node == '1'
run: |
curl --silent https://unofficial-builds.nodejs.org/download/release/v20.18.1/node-v20.18.1-linux-x64-glibc-217.tar.xz | tar -xJ --strip-components 1 -C /node20217 -f -
# We would like to use harden-runner, but it flags too many false
# positives, every time we download a dependency. We should use it only
# on CI runs where we are producing artifacts that users might rely on.
Expand Down

0 comments on commit 7179e2c

Please sign in to comment.